[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:16:02.678460 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:39131
I20260709 12:16:02.680383 24421 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:16:02.681576 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:02.701685 24430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:02.701733 24427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:02.701769 24428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:03.905838 24429 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 12:16:03.906071 24421 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:16:03.911056 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:03.911316 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:03.911535 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599363911510 us; error 0 us; skew 500 ppm
I20260709 12:16:03.912398 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:03.918991 24421 webserver.cc:533] Webserver started at http://127.23.217.126:35609/ using document root <none> and password file <none>
I20260709 12:16:03.919894 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:03.920145 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:03.920622 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:03.926657 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "bde9f1b1bd88476cbac796c31003312f"
format_stamp: "Formatted at 2026-07-09 12:16:03 on dist-test-slave-lg8f"
I20260709 12:16:03.935155 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:16:03.940742 24436 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:03.942094 24421 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:16:03.942528 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "bde9f1b1bd88476cbac796c31003312f"
format_stamp: "Formatted at 2026-07-09 12:16:03 on dist-test-slave-lg8f"
I20260709 12:16:03.942904 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:03.994743 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:03.996352 24421 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:16:03.996838 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:04.073693 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:39131
I20260709 12:16:04.073891 24487 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:39131 every 8 connection(s)
I20260709 12:16:04.082834 24488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:04.107087 24488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f: Bootstrap starting.
I20260709 12:16:04.114130 24488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.116647 24488 log.cc:826] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f: Log is configured to *not* fsync() on all Append() calls
I20260709 12:16:04.123100 24488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f: No bootstrap required, opened a new log
I20260709 12:16:04.145614 24488 raft_consensus.cc:359] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde9f1b1bd88476cbac796c31003312f" member_type: VOTER }
I20260709 12:16:04.146262 24488 raft_consensus.cc:385] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.146600 24488 raft_consensus.cc:740] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde9f1b1bd88476cbac796c31003312f, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.147776 24488 consensus_queue.cc:260] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde9f1b1bd88476cbac796c31003312f" member_type: VOTER }
I20260709 12:16:04.148339 24488 raft_consensus.cc:399] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:04.148690 24488 raft_consensus.cc:493] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:04.149212 24488 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.154691 24488 raft_consensus.cc:515] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde9f1b1bd88476cbac796c31003312f" member_type: VOTER }
I20260709 12:16:04.155547 24488 leader_election.cc:304] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bde9f1b1bd88476cbac796c31003312f; no voters: 
I20260709 12:16:04.155632 24491 raft_consensus.cc:493] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.156128 24491 raft_consensus.cc:515] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde9f1b1bd88476cbac796c31003312f" member_type: VOTER }
I20260709 12:16:04.158691 24488 leader_election.cc:290] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:04.160066 24492 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:04.163726 24492 raft_consensus.cc:697] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 1 LEADER]: Becoming Leader. State: Replica: bde9f1b1bd88476cbac796c31003312f, State: Running, Role: LEADER
I20260709 12:16:04.163936 24491 leader_election.cc:304] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bde9f1b1bd88476cbac796c31003312f; no voters: 
I20260709 12:16:04.164747 24492 consensus_queue.cc:237] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde9f1b1bd88476cbac796c31003312f" member_type: VOTER }
I20260709 12:16:04.165275 24488 sys_catalog.cc:565] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:04.166531 24491 leader_election.cc:290] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 
I20260709 12:16:04.172530 24493 sys_catalog.cc:455] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bde9f1b1bd88476cbac796c31003312f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde9f1b1bd88476cbac796c31003312f" member_type: VOTER } }
I20260709 12:16:04.174945 24493 sys_catalog.cc:458] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:04.173880 24494 sys_catalog.cc:455] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bde9f1b1bd88476cbac796c31003312f. Latest consensus state: current_term: 1 leader_uuid: "bde9f1b1bd88476cbac796c31003312f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bde9f1b1bd88476cbac796c31003312f" member_type: VOTER } }
I20260709 12:16:04.177088 24494 sys_catalog.cc:458] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:04.182148 24501 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:04.194433 24501 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:04.199314 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:04.213965 24501 catalog_manager.cc:1383] Generated new cluster ID: 1d88ef1d87b9452aa2a32bab07e6ddf9
I20260709 12:16:04.214308 24501 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:04.234771 24501 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:04.236398 24501 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:04.256084 24501 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f: Generated new TSK 0
I20260709 12:16:04.257064 24501 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:04.266000 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:04.275121 24512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:04.277065 24513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:04.281780 24421 server_base.cc:1061] running on GCE node
W20260709 12:16:04.282014 24515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:04.283077 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:04.283278 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:04.283651 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599364283611 us; error 0 us; skew 500 ppm
I20260709 12:16:04.284307 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:04.288667 24421 webserver.cc:533] Webserver started at http://127.23.217.65:36409/ using document root <none> and password file <none>
I20260709 12:16:04.289239 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:04.289474 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:04.289917 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:04.291855 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "b1c10cabe1f349938525b635ca28c4f3"
format_stamp: "Formatted at 2026-07-09 12:16:04 on dist-test-slave-lg8f"
I20260709 12:16:04.298295 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.005s
I20260709 12:16:04.302910 24520 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.303830 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:16:04.304155 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "b1c10cabe1f349938525b635ca28c4f3"
format_stamp: "Formatted at 2026-07-09 12:16:04 on dist-test-slave-lg8f"
I20260709 12:16:04.304467 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:04.327521 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:04.328900 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:04.331736 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:04.335958 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:04.336217 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.336481 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:04.336683 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.444914 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:45443
I20260709 12:16:04.445065 24582 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:45443 every 8 connection(s)
I20260709 12:16:04.452131 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:04.462947 24587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:04.467592 24588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:04.473891 24421 server_base.cc:1061] running on GCE node
W20260709 12:16:04.474210 24590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:04.475371 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:04.475571 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:04.475807 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599364475772 us; error 0 us; skew 500 ppm
I20260709 12:16:04.476531 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:04.485060 24583 heartbeater.cc:344] Connected to a master server at 127.23.217.126:39131
I20260709 12:16:04.485621 24583 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:04.486954 24583 heartbeater.cc:507] Master 127.23.217.126:39131 requested a full tablet report, sending...
I20260709 12:16:04.487313 24421 webserver.cc:533] Webserver started at http://127.23.217.66:45505/ using document root <none> and password file <none>
I20260709 12:16:04.487888 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:04.488124 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:04.488417 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:04.489585 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
format_stamp: "Formatted at 2026-07-09 12:16:04 on dist-test-slave-lg8f"
I20260709 12:16:04.490927 24453 ts_manager.cc:194] Registered new tserver with Master: b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443)
I20260709 12:16:04.494292 24453 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36192
I20260709 12:16:04.495419 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20260709 12:16:04.500090 24595 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.500967 24421 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:16:04.501250 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
format_stamp: "Formatted at 2026-07-09 12:16:04 on dist-test-slave-lg8f"
I20260709 12:16:04.501503 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:04.521248 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:04.522575 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:04.524792 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:04.528156 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:04.528329 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.528630 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:04.528769 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.633631 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:40233
I20260709 12:16:04.633767 24657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:40233 every 8 connection(s)
I20260709 12:16:04.640993 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:04.650153 24661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:04.652279 24662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:04.660373 24658 heartbeater.cc:344] Connected to a master server at 127.23.217.126:39131
I20260709 12:16:04.660882 24658 heartbeater.cc:461] Registering TS with master...
W20260709 12:16:04.661437 24664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:04.662453 24658 heartbeater.cc:507] Master 127.23.217.126:39131 requested a full tablet report, sending...
I20260709 12:16:04.662899 24421 server_base.cc:1061] running on GCE node
I20260709 12:16:04.663969 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:04.664219 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:04.664426 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599364664406 us; error 0 us; skew 500 ppm
I20260709 12:16:04.665040 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:04.665076 24453 ts_manager.cc:194] Registered new tserver with Master: ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:04.666883 24453 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36200
I20260709 12:16:04.668322 24421 webserver.cc:533] Webserver started at http://127.23.217.67:38611/ using document root <none> and password file <none>
I20260709 12:16:04.668995 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:04.669237 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:04.669543 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:04.670887 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "750cf88774294edb98402c769337d105"
format_stamp: "Formatted at 2026-07-09 12:16:04 on dist-test-slave-lg8f"
I20260709 12:16:04.675902 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 12:16:04.679695 24669 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.680688 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:04.681011 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "750cf88774294edb98402c769337d105"
format_stamp: "Formatted at 2026-07-09 12:16:04 on dist-test-slave-lg8f"
I20260709 12:16:04.681339 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:04.705884 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:04.707410 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:04.709031 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:04.711712 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:04.711959 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.712212 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:04.712404 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:04.827440 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:37701
I20260709 12:16:04.828146 24731 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:37701 every 8 connection(s)
I20260709 12:16:04.848424 24732 heartbeater.cc:344] Connected to a master server at 127.23.217.126:39131
I20260709 12:16:04.848925 24732 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:04.849917 24732 heartbeater.cc:507] Master 127.23.217.126:39131 requested a full tablet report, sending...
I20260709 12:16:04.852391 24453 ts_manager.cc:194] Registered new tserver with Master: 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:04.853399 24421 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017099739s
I20260709 12:16:04.853926 24453 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36210
I20260709 12:16:04.913729 24453 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36214:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:04.917866 24453 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:05.029666 24695 tablet_service.cc:1511] Processing CreateTablet for tablet 912720e8d5b644e4bceda48a449b9b8d (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:05.031366 24696 tablet_service.cc:1511] Processing CreateTablet for tablet a91da17b78bd4f188511e5ae1ee51da5 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:05.029604 24697 tablet_service.cc:1511] Processing CreateTablet for tablet 3af595ba369a4e15a50274c854c116bf (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:05.031819 24695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912720e8d5b644e4bceda48a449b9b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.034976 24696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91da17b78bd4f188511e5ae1ee51da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.035811 24697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af595ba369a4e15a50274c854c116bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.043270 24621 tablet_service.cc:1511] Processing CreateTablet for tablet 912720e8d5b644e4bceda48a449b9b8d (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:05.043620 24623 tablet_service.cc:1511] Processing CreateTablet for tablet 3af595ba369a4e15a50274c854c116bf (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:05.044039 24622 tablet_service.cc:1511] Processing CreateTablet for tablet a91da17b78bd4f188511e5ae1ee51da5 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:05.049361 24620 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5b4d278c17437ab3c075b769910253 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:05.052531 24621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912720e8d5b644e4bceda48a449b9b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.054399 24622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91da17b78bd4f188511e5ae1ee51da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.056674 24620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5b4d278c17437ab3c075b769910253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.060179 24694 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5b4d278c17437ab3c075b769910253 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:05.062384 24623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af595ba369a4e15a50274c854c116bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.063661 24694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5b4d278c17437ab3c075b769910253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.065647 24619 tablet_service.cc:1511] Processing CreateTablet for tablet 04d91e2c217b4008bc9643cccd37fe93 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:05.071870 24693 tablet_service.cc:1511] Processing CreateTablet for tablet 04d91e2c217b4008bc9643cccd37fe93 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:05.075030 24693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d91e2c217b4008bc9643cccd37fe93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.073691 24619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d91e2c217b4008bc9643cccd37fe93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.085240 24618 tablet_service.cc:1511] Processing CreateTablet for tablet 5be7a7bc10504a6586d316a44947410a (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:05.089968 24618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5be7a7bc10504a6586d316a44947410a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.114625 24692 tablet_service.cc:1511] Processing CreateTablet for tablet 5be7a7bc10504a6586d316a44947410a (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:05.116675 24692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5be7a7bc10504a6586d316a44947410a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.133030 24548 tablet_service.cc:1511] Processing CreateTablet for tablet 3af595ba369a4e15a50274c854c116bf (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:05.135381 24545 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5b4d278c17437ab3c075b769910253 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:05.135898 24546 tablet_service.cc:1511] Processing CreateTablet for tablet 912720e8d5b644e4bceda48a449b9b8d (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:05.138016 24543 tablet_service.cc:1511] Processing CreateTablet for tablet 5be7a7bc10504a6586d316a44947410a (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:05.138357 24544 tablet_service.cc:1511] Processing CreateTablet for tablet 04d91e2c217b4008bc9643cccd37fe93 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:05.133507 24547 tablet_service.cc:1511] Processing CreateTablet for tablet a91da17b78bd4f188511e5ae1ee51da5 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:05.139679 24548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af595ba369a4e15a50274c854c116bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.140504 24547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91da17b78bd4f188511e5ae1ee51da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.141166 24546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912720e8d5b644e4bceda48a449b9b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.142827 24544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d91e2c217b4008bc9643cccd37fe93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.143488 24543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5be7a7bc10504a6586d316a44947410a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.157213 24545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5b4d278c17437ab3c075b769910253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.168087 24617 tablet_service.cc:1511] Processing CreateTablet for tablet 1b20d8cc118049678c94d8d489e3caca (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:05.169394 24617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b20d8cc118049678c94d8d489e3caca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.154050 24691 tablet_service.cc:1511] Processing CreateTablet for tablet 1b20d8cc118049678c94d8d489e3caca (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:05.172911 24691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b20d8cc118049678c94d8d489e3caca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.176692 24542 tablet_service.cc:1511] Processing CreateTablet for tablet 1b20d8cc118049678c94d8d489e3caca (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:05.203698 24541 tablet_service.cc:1511] Processing CreateTablet for tablet d61a903607af4301a7a84013e37809b0 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:05.218760 24542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b20d8cc118049678c94d8d489e3caca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.222416 24750 tablet_bootstrap.cc:492] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:05.229463 24616 tablet_service.cc:1511] Processing CreateTablet for tablet d61a903607af4301a7a84013e37809b0 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:05.236905 24541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d61a903607af4301a7a84013e37809b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.244760 24616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d61a903607af4301a7a84013e37809b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.243453 24690 tablet_service.cc:1511] Processing CreateTablet for tablet d61a903607af4301a7a84013e37809b0 (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
W20260709 12:16:05.250041 24503 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:05.248440 24690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d61a903607af4301a7a84013e37809b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.264353 24540 tablet_service.cc:1511] Processing CreateTablet for tablet 0c146994b1d8431bad2e830d387693fe (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:05.267174 24751 tablet_bootstrap.cc:492] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:05.267282 24540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c146994b1d8431bad2e830d387693fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.311364 24619 tablet_service.cc:1511] Processing CreateTablet for tablet 0c146994b1d8431bad2e830d387693fe (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:05.312953 24619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c146994b1d8431bad2e830d387693fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.400388 24752 tablet_bootstrap.cc:492] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.408543 24751 tablet_bootstrap.cc:654] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.420365 24693 tablet_service.cc:1511] Processing CreateTablet for tablet 0c146994b1d8431bad2e830d387693fe (DEFAULT_TABLE table=test-workload [id=c40cfa6d953c4be19434da39d9c82560]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:05.422971 24750 tablet_bootstrap.cc:654] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.424039 24693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c146994b1d8431bad2e830d387693fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:05.431566 24752 tablet_bootstrap.cc:654] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.441326 24751 tablet_bootstrap.cc:492] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:05.441823 24752 tablet_bootstrap.cc:492] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:05.441969 24751 ts_tablet_manager.cc:1403] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.193s	user 0.011s	sys 0.007s
I20260709 12:16:05.442312 24752 ts_tablet_manager.cc:1403] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.000s
I20260709 12:16:05.446362 24752 raft_consensus.cc:359] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.446223 24751 raft_consensus.cc:359] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:05.447183 24752 raft_consensus.cc:385] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.447288 24751 raft_consensus.cc:385] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.447570 24752 raft_consensus.cc:740] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.447644 24751 raft_consensus.cc:740] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.448529 24752 consensus_queue.cc:260] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.448688 24751 consensus_queue.cc:260] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:05.454983 24750 tablet_bootstrap.cc:492] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:05.455504 24750 ts_tablet_manager.cc:1403] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.234s	user 0.012s	sys 0.015s
I20260709 12:16:05.458681 24750 raft_consensus.cc:359] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.459152 24751 ts_tablet_manager.cc:1434] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.006s
I20260709 12:16:05.459417 24750 raft_consensus.cc:385] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.459867 24750 raft_consensus.cc:740] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.460032 24583 heartbeater.cc:499] Master 127.23.217.126:39131 was elected leader, sending a full tablet report...
I20260709 12:16:05.461127 24732 heartbeater.cc:499] Master 127.23.217.126:39131 was elected leader, sending a full tablet report...
I20260709 12:16:05.460552 24751 tablet_bootstrap.cc:492] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:05.460636 24750 consensus_queue.cc:260] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.470410 24751 tablet_bootstrap.cc:654] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.479604 24658 heartbeater.cc:499] Master 127.23.217.126:39131 was elected leader, sending a full tablet report...
I20260709 12:16:05.459240 24752 ts_tablet_manager.cc:1434] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.010s
I20260709 12:16:05.487685 24750 ts_tablet_manager.cc:1434] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.014s
I20260709 12:16:05.481323 24752 tablet_bootstrap.cc:492] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.488798 24750 tablet_bootstrap.cc:492] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:05.493400 24751 tablet_bootstrap.cc:492] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:05.493983 24751 ts_tablet_manager.cc:1403] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.012s
I20260709 12:16:05.497164 24750 tablet_bootstrap.cc:654] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.497589 24751 raft_consensus.cc:359] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.498549 24751 raft_consensus.cc:385] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.499027 24751 raft_consensus.cc:740] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.500116 24751 consensus_queue.cc:260] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.501093 24752 tablet_bootstrap.cc:654] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.502900 24751 ts_tablet_manager.cc:1434] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.005s
I20260709 12:16:05.504024 24751 tablet_bootstrap.cc:492] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:05.511695 24751 tablet_bootstrap.cc:654] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.517630 24750 tablet_bootstrap.cc:492] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:05.518193 24750 ts_tablet_manager.cc:1403] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.003s
I20260709 12:16:05.521590 24750 raft_consensus.cc:359] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.522428 24750 raft_consensus.cc:385] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.523010 24750 raft_consensus.cc:740] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.523926 24750 consensus_queue.cc:260] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.526404 24750 ts_tablet_manager.cc:1434] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:16:05.527451 24750 tablet_bootstrap.cc:492] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:05.533424 24756 raft_consensus.cc:493] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.534016 24756 raft_consensus.cc:515] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.534062 24752 tablet_bootstrap.cc:492] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:05.535674 24752 ts_tablet_manager.cc:1403] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.026s
I20260709 12:16:05.537844 24756 leader_election.cc:290] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:05.544064 24750 tablet_bootstrap.cc:654] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.545425 24752 raft_consensus.cc:359] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.546211 24752 raft_consensus.cc:385] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.549026 24752 raft_consensus.cc:740] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.549160 24751 tablet_bootstrap.cc:492] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:05.549671 24751 ts_tablet_manager.cc:1403] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.003s
I20260709 12:16:05.552495 24751 raft_consensus.cc:359] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.553182 24751 raft_consensus.cc:385] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.553519 24751 raft_consensus.cc:740] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.554250 24751 consensus_queue.cc:260] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.556375 24751 ts_tablet_manager.cc:1434] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:05.557495 24751 tablet_bootstrap.cc:492] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:05.560468 24752 consensus_queue.cc:260] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.571374 24752 ts_tablet_manager.cc:1434] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:16:05.572376 24752 tablet_bootstrap.cc:492] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.578868 24751 tablet_bootstrap.cc:654] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:05.588841 24733 tablet.cc:2406] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:05.599373 24752 tablet_bootstrap.cc:654] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.617986 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:05.618916 24558 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
I20260709 12:16:05.620563 24670 leader_election.cc:304] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:05.621467 24756 raft_consensus.cc:2804] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.621964 24756 raft_consensus.cc:493] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.622354 24756 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.623929 24752 tablet_bootstrap.cc:492] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:05.624423 24752 ts_tablet_manager.cc:1403] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.002s
I20260709 12:16:05.627593 24752 raft_consensus.cc:359] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.628316 24752 raft_consensus.cc:385] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.628614 24752 raft_consensus.cc:740] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.628811 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
I20260709 12:16:05.629369 24752 consensus_queue.cc:260] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.630620 24756 raft_consensus.cc:515] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
W20260709 12:16:05.631362 24670 leader_election.cc:343] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.632659 24756 leader_election.cc:290] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:05.633682 24751 tablet_bootstrap.cc:492] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:05.634335 24751 ts_tablet_manager.cc:1403] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.077s	user 0.013s	sys 0.003s
I20260709 12:16:05.635689 24750 tablet_bootstrap.cc:492] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:05.635891 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:05.636292 24750 ts_tablet_manager.cc:1403] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.109s	user 0.046s	sys 0.039s
I20260709 12:16:05.636224 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:05.636525 24558 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:05.637705 24670 leader_election.cc:343] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.641420 24752 ts_tablet_manager.cc:1434] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:16:05.641402 24751 raft_consensus.cc:359] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.642241 24751 raft_consensus.cc:385] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.642551 24751 raft_consensus.cc:740] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.642628 24752 tablet_bootstrap.cc:492] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.643263 24751 consensus_queue.cc:260] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.645401 24751 ts_tablet_manager.cc:1434] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:16:05.646317 24751 tablet_bootstrap.cc:492] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:05.653085 24751 tablet_bootstrap.cc:654] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.653424 24752 tablet_bootstrap.cc:654] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.669232 24751 tablet_bootstrap.cc:492] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:05.669732 24751 ts_tablet_manager.cc:1403] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 12:16:05.671402 24558 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:05.672360 24752 tablet_bootstrap.cc:492] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:05.671994 24750 raft_consensus.cc:359] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.672781 24750 raft_consensus.cc:385] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.672894 24752 ts_tablet_manager.cc:1403] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.007s
I20260709 12:16:05.672871 24670 leader_election.cc:304] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:05.672778 24751 raft_consensus.cc:359] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.673957 24756 raft_consensus.cc:2804] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:05.673240 24750 raft_consensus.cc:740] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.674255 24751 raft_consensus.cc:385] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.674803 24751 raft_consensus.cc:740] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.675321 24750 consensus_queue.cc:260] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.675637 24751 consensus_queue.cc:260] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.676474 24752 raft_consensus.cc:359] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:05.677388 24752 raft_consensus.cc:385] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.677886 24750 ts_tablet_manager.cc:1434] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:16:05.677779 24752 raft_consensus.cc:740] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.679028 24750 tablet_bootstrap.cc:492] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:05.679294 24752 consensus_queue.cc:260] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:05.682170 24752 ts_tablet_manager.cc:1434] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:16:05.683842 24752 tablet_bootstrap.cc:492] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.686585 24756 raft_consensus.cc:697] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:05.687553 24756 consensus_queue.cc:237] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.690107 24752 tablet_bootstrap.cc:654] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.692402 24750 tablet_bootstrap.cc:654] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.704581 24752 tablet_bootstrap.cc:492] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:05.705089 24752 ts_tablet_manager.cc:1403] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.021s	user 0.001s	sys 0.016s
I20260709 12:16:05.707937 24752 raft_consensus.cc:359] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.708649 24752 raft_consensus.cc:385] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.708940 24752 raft_consensus.cc:740] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.709630 24752 consensus_queue.cc:260] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.712045 24752 ts_tablet_manager.cc:1434] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:05.713310 24752 tablet_bootstrap.cc:492] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.717639 24453 catalog_manager.cc:5697] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 0 to 1, leader changed from <none> to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.719426 24752 tablet_bootstrap.cc:654] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.735960 24751 ts_tablet_manager.cc:1434] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.066s	user 0.017s	sys 0.009s
I20260709 12:16:05.736958 24751 tablet_bootstrap.cc:492] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:05.742794 24750 tablet_bootstrap.cc:492] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:05.743270 24752 tablet_bootstrap.cc:492] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:05.743628 24750 ts_tablet_manager.cc:1403] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.065s	user 0.013s	sys 0.018s
I20260709 12:16:05.743767 24752 ts_tablet_manager.cc:1403] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.005s
I20260709 12:16:05.747226 24751 tablet_bootstrap.cc:654] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.751657 24752 raft_consensus.cc:359] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.753783 24752 raft_consensus.cc:385] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.754372 24758 raft_consensus.cc:493] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.755223 24752 raft_consensus.cc:740] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.755542 24750 raft_consensus.cc:359] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.761915 24750 raft_consensus.cc:385] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.762411 24750 raft_consensus.cc:740] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.755126 24758 raft_consensus.cc:515] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.763767 24750 consensus_queue.cc:260] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.768275 24758 leader_election.cc:290] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:05.770440 24758 raft_consensus.cc:493] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.768878 24752 consensus_queue.cc:260] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.771409 24758 raft_consensus.cc:515] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.776386 24752 ts_tablet_manager.cc:1434] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.004s
I20260709 12:16:05.777437 24752 tablet_bootstrap.cc:492] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.780006 24758 leader_election.cc:290] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:05.785588 24750 ts_tablet_manager.cc:1434] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.010s
I20260709 12:16:05.786870 24750 tablet_bootstrap.cc:492] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:05.795816 24752 tablet_bootstrap.cc:654] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.797613 24775 raft_consensus.cc:493] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.798235 24775 raft_consensus.cc:515] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:05.800634 24775 leader_election.cc:290] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:05.800983 24751 tablet_bootstrap.cc:492] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:05.801518 24751 ts_tablet_manager.cc:1403] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.012s
I20260709 12:16:05.802718 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:05.803305 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
W20260709 12:16:05.804811 24670 leader_election.cc:343] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.803416 24558 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
I20260709 12:16:05.805107 24751 raft_consensus.cc:359] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.805831 24751 raft_consensus.cc:385] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.806146 24751 raft_consensus.cc:740] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.806565 24670 leader_election.cc:304] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:05.806972 24751 consensus_queue.cc:260] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.807405 24775 raft_consensus.cc:2804] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.808063 24775 raft_consensus.cc:493] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.808420 24775 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.809202 24757 raft_consensus.cc:493] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.810775 24757 raft_consensus.cc:515] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.815907 24751 ts_tablet_manager.cc:1434] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 12:16:05.816922 24751 tablet_bootstrap.cc:492] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:05.817516 24750 tablet_bootstrap.cc:654] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.823617 24757 leader_election.cc:290] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:05.847306 24775 raft_consensus.cc:515] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:05.858075 24781 raft_consensus.cc:493] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.859167 24781 raft_consensus.cc:515] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.872581 24781 leader_election.cc:290] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:05.874038 24775 leader_election.cc:290] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:05.874779 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d91e2c217b4008bc9643cccd37fe93" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:05.875219 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:05.876085 24751 tablet_bootstrap.cc:654] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.885481 24557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:05.887377 24557 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 0.
I20260709 12:16:05.887493 24556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
W20260709 12:16:05.892019 24670 leader_election.cc:343] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.892009 24556 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 0.
I20260709 12:16:05.893555 24596 leader_election.cc:304] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:05.884886 24558 raft_consensus.cc:2468] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 0.
I20260709 12:16:05.899096 24596 leader_election.cc:304] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:05.900246 24758 raft_consensus.cc:2804] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.900379 24596 leader_election.cc:304] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:05.900821 24758 raft_consensus.cc:493] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.901292 24758 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.904228 24782 raft_consensus.cc:2804] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.903559 24781 raft_consensus.cc:2804] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.905169 24782 raft_consensus.cc:493] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.905206 24781 raft_consensus.cc:493] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.905977 24782 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.906090 24781 raft_consensus.cc:3060] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.911913 24775 raft_consensus.cc:493] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.913801 24775 raft_consensus.cc:515] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.915539 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d91e2c217b4008bc9643cccd37fe93" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105" is_pre_election: true
I20260709 12:16:05.916177 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105" is_pre_election: true
I20260709 12:16:05.916556 24707 raft_consensus.cc:2468] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 0.
I20260709 12:16:05.918735 24705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105" is_pre_election: true
I20260709 12:16:05.920142 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:05.921377 24558 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.917846 24706 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 0.
I20260709 12:16:05.921388 24705 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 0.
I20260709 12:16:05.923673 24752 tablet_bootstrap.cc:492] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:05.924185 24752 ts_tablet_manager.cc:1403] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.147s	user 0.004s	sys 0.013s
I20260709 12:16:05.925424 24556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:05.926437 24556 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
I20260709 12:16:05.927944 24670 leader_election.cc:304] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:05.928643 24775 leader_election.cc:290] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:05.929318 24775 raft_consensus.cc:2804] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.929049 24752 raft_consensus.cc:359] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.929678 24775 raft_consensus.cc:493] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.930107 24775 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.930265 24752 raft_consensus.cc:385] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.930562 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
I20260709 12:16:05.930763 24752 raft_consensus.cc:740] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.931712 24752 consensus_queue.cc:260] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.938902 24558 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:05.939040 24758 raft_consensus.cc:515] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.942409 24752 ts_tablet_manager.cc:1434] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 12:16:05.946616 24670 leader_election.cc:304] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:05.947942 24756 raft_consensus.cc:2804] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:05.948968 24633 raft_consensus.cc:2393] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 750cf88774294edb98402c769337d105 in current term 1: Already voted for candidate ceb6ff4f382f459d9b6b29417d6ff574 in this term.
I20260709 12:16:05.948985 24756 raft_consensus.cc:697] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:05.949465 24752 tablet_bootstrap.cc:492] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:05.950182 24756 consensus_queue.cc:237] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:05.952502 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:05.954149 24558 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.953625 24750 tablet_bootstrap.cc:492] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:05.955348 24750 ts_tablet_manager.cc:1403] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.169s	user 0.006s	sys 0.029s
I20260709 12:16:05.958938 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:05.954917 24758 leader_election.cc:290] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:05.967348 24705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c146994b1d8431bad2e830d387693fe" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105" is_pre_election: true
I20260709 12:16:05.968127 24775 raft_consensus.cc:515] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.969199 24782 raft_consensus.cc:515] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
W20260709 12:16:05.973169 24523 leader_election.cc:343] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 750cf88774294edb98402c769337d105 (127.23.217.67:37701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.971071 24781 raft_consensus.cc:515] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:05.974603 24751 tablet_bootstrap.cc:492] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:05.975996 24751 ts_tablet_manager.cc:1403] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.159s	user 0.010s	sys 0.007s
I20260709 12:16:05.976222 24706 raft_consensus.cc:2393] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ceb6ff4f382f459d9b6b29417d6ff574 in current term 1: Already voted for candidate 750cf88774294edb98402c769337d105 in this term.
I20260709 12:16:05.982406 24557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:05.982817 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c146994b1d8431bad2e830d387693fe" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
I20260709 12:16:05.983038 24556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:05.984157 24782 leader_election.cc:290] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:05.984998 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d91e2c217b4008bc9643cccd37fe93" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:05.985224 24556 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.984205 24775 leader_election.cc:290] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:05.986245 24781 leader_election.cc:290] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:05.987192 24555 raft_consensus.cc:3060] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.990213 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:05.992449 24705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:05.993042 24705 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.993614 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d91e2c217b4008bc9643cccd37fe93" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105"
W20260709 12:16:05.994886 24521 leader_election.cc:343] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.995288 24707 raft_consensus.cc:3060] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.996675 24521 leader_election.cc:304] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3; no voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:06.001314 24757 raft_consensus.cc:2749] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:05.998456 24558 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 1.
I20260709 12:16:06.004050 24750 raft_consensus.cc:359] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.005182 24750 raft_consensus.cc:385] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.007102 24750 raft_consensus.cc:740] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.005966 24670 leader_election.cc:304] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 750cf88774294edb98402c769337d105; no voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:06.007615 24596 leader_election.cc:304] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:06.008597 24556 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 1.
I20260709 12:16:06.009500 24705 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 1.
I20260709 12:16:06.011163 24781 raft_consensus.cc:2804] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.011596 24775 raft_consensus.cc:2749] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.011736 24781 raft_consensus.cc:697] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:06.012045 24596 leader_election.cc:304] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:06.011922 24751 raft_consensus.cc:359] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.012856 24751 raft_consensus.cc:385] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.013125 24758 raft_consensus.cc:2804] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.013283 24751 raft_consensus.cc:740] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.013648 24758 raft_consensus.cc:697] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:06.012696 24781 consensus_queue.cc:237] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.014086 24751 consensus_queue.cc:260] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.014528 24758 consensus_queue.cc:237] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.009883 24752 tablet_bootstrap.cc:654] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.016695 24751 ts_tablet_manager.cc:1434] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:16:06.017417 24555 raft_consensus.cc:2468] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 1.
I20260709 12:16:06.018586 24596 leader_election.cc:304] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:06.019482 24782 raft_consensus.cc:2804] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.017720 24751 tablet_bootstrap.cc:492] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:06.014837 24707 raft_consensus.cc:2468] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 1.
I20260709 12:16:06.042232 24751 tablet_bootstrap.cc:654] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.022761 24750 consensus_queue.cc:260] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.046270 24453 catalog_manager.cc:5697] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 0 to 1, leader changed from <none> to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:06.047658 24750 ts_tablet_manager.cc:1434] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.091s	user 0.005s	sys 0.005s
I20260709 12:16:06.048671 24750 tablet_bootstrap.cc:492] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:06.049576 24775 raft_consensus.cc:493] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.050105 24775 raft_consensus.cc:515] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.062232 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
W20260709 12:16:06.064137 24670 leader_election.cc:343] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.060323 24782 raft_consensus.cc:697] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:06.068100 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
I20260709 12:16:06.069805 24633 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
I20260709 12:16:06.069859 24782 consensus_queue.cc:237] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.071748 24670 leader_election.cc:304] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574; no voters: b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.068222 24775 leader_election.cc:290] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.072919 24756 raft_consensus.cc:2804] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:06.074292 24756 raft_consensus.cc:493] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.076120 24756 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.080654 24752 tablet_bootstrap.cc:492] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:06.081254 24752 ts_tablet_manager.cc:1403] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.132s	user 0.022s	sys 0.011s
I20260709 12:16:06.085461 24751 tablet_bootstrap.cc:492] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:06.085875 24751 ts_tablet_manager.cc:1403] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.006s
I20260709 12:16:06.087963 24752 raft_consensus.cc:359] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.088701 24752 raft_consensus.cc:385] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.075366 24452 catalog_manager.cc:5697] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:06.089072 24752 raft_consensus.cc:740] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.091290 24633 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.092110 24750 tablet_bootstrap.cc:654] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.089782 24752 consensus_queue.cc:260] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.094671 24751 raft_consensus.cc:359] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.095095 24752 ts_tablet_manager.cc:1434] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 12:16:06.095312 24751 raft_consensus.cc:385] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.096503 24751 raft_consensus.cc:740] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.097237 24752 tablet_bootstrap.cc:492] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3: Bootstrap starting.
I20260709 12:16:06.099995 24751 consensus_queue.cc:260] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.106819 24775 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:06.114970 24751 ts_tablet_manager.cc:1434] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.029s	user 0.015s	sys 0.002s
I20260709 12:16:06.117427 24751 tablet_bootstrap.cc:492] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105: Bootstrap starting.
I20260709 12:16:06.114676 24756 raft_consensus.cc:515] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.174094 24756 leader_election.cc:290] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.174723 24556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:06.177219 24556 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.177799 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:06.180325 24632 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.181686 24782 raft_consensus.cc:493] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.183250 24782 raft_consensus.cc:515] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.186204 24751 tablet_bootstrap.cc:654] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.185719 24756 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:06.189352 24750 tablet_bootstrap.cc:492] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:06.190943 24750 ts_tablet_manager.cc:1403] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.142s	user 0.011s	sys 0.009s
I20260709 12:16:06.196201 24752 tablet_bootstrap.cc:654] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.199707 24556 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:06.201220 24670 leader_election.cc:304] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:06.201278 24632 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:06.211766 24756 raft_consensus.cc:2804] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.214033 24756 raft_consensus.cc:697] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:06.219707 24750 raft_consensus.cc:359] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:06.221899 24750 raft_consensus.cc:385] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.222210 24750 raft_consensus.cc:740] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.222604 24752 tablet_bootstrap.cc:492] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3: No bootstrap required, opened a new log
I20260709 12:16:06.223071 24752 ts_tablet_manager.cc:1403] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3: Time spent bootstrapping tablet: real 0.126s	user 0.013s	sys 0.005s
I20260709 12:16:06.223213 24750 consensus_queue.cc:260] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:06.224185 24756 consensus_queue.cc:237] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.228462 24750 ts_tablet_manager.cc:1434] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:16:06.230585 24750 tablet_bootstrap.cc:492] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:06.232065 24782 leader_election.cc:290] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:06.233006 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105" is_pre_election: true
I20260709 12:16:06.234984 24751 tablet_bootstrap.cc:492] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105: No bootstrap required, opened a new log
I20260709 12:16:06.235447 24751 ts_tablet_manager.cc:1403] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105: Time spent bootstrapping tablet: real 0.118s	user 0.030s	sys 0.026s
I20260709 12:16:06.236268 24752 raft_consensus.cc:359] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.236792 24752 raft_consensus.cc:385] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.237022 24752 raft_consensus.cc:740] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.239162 24752 consensus_queue.cc:260] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.240567 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:06.273701 24775 raft_consensus.cc:493] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.274314 24775 raft_consensus.cc:515] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.275245 24596 leader_election.cc:304] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb6ff4f382f459d9b6b29417d6ff574; no voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.281332 24771 raft_consensus.cc:493] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.281502 24752 ts_tablet_manager.cc:1434] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 12:16:06.283962 24771 raft_consensus.cc:515] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.282289 24782 raft_consensus.cc:2749] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:06.288713 24771 leader_election.cc:290] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.289907 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:06.290619 24555 raft_consensus.cc:2468] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
W20260709 12:16:06.298702 24503 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:06.299443 24670 leader_election.cc:304] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:06.300631 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
I20260709 12:16:06.302657 24771 raft_consensus.cc:2804] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:06.303287 24771 raft_consensus.cc:493] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.303685 24771 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.313113 24771 raft_consensus.cc:515] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.301865 24451 catalog_manager.cc:5697] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:06.323433 24451 catalog_manager.cc:5697] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:06.330787 24771 leader_election.cc:290] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.333894 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
W20260709 12:16:06.335457 24670 leader_election.cc:343] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.340199 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:06.339780 24555 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.356052 24556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:06.356660 24556 raft_consensus.cc:2468] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
I20260709 12:16:06.358590 24751 raft_consensus.cc:359] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.359380 24751 raft_consensus.cc:385] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.359730 24751 raft_consensus.cc:740] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.359942 24750 tablet_bootstrap.cc:654] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.360644 24751 consensus_queue.cc:260] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.362860 24751 ts_tablet_manager.cc:1434] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105: Time spent starting tablet: real 0.127s	user 0.010s	sys 0.023s
I20260709 12:16:06.362900 24775 leader_election.cc:290] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.364991 24453 catalog_manager.cc:5697] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 0 to 1, leader changed from <none> to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:06.358180 24670 leader_election.cc:343] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.368531 24555 raft_consensus.cc:2468] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:06.369374 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
I20260709 12:16:06.371491 24670 leader_election.cc:304] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:06.372264 24756 raft_consensus.cc:2804] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:06.372460 24670 leader_election.cc:304] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:06.372691 24756 raft_consensus.cc:493] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.373101 24756 raft_consensus.cc:3060] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.373276 24775 raft_consensus.cc:2804] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.373867 24775 raft_consensus.cc:697] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:06.374706 24775 consensus_queue.cc:237] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
W20260709 12:16:06.375074 24670 leader_election.cc:343] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.381053 24750 tablet_bootstrap.cc:492] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:06.381603 24750 ts_tablet_manager.cc:1403] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.151s	user 0.013s	sys 0.016s
I20260709 12:16:06.383862 24756 raft_consensus.cc:515] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.386272 24756 leader_election.cc:290] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.387261 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:06.387674 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:06.388134 24555 raft_consensus.cc:3060] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:06.389268 24670 leader_election.cc:343] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.389418 24750 raft_consensus.cc:359] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.390169 24750 raft_consensus.cc:385] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.390516 24750 raft_consensus.cc:740] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.391310 24750 consensus_queue.cc:260] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.394472 24750 ts_tablet_manager.cc:1434] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:16:06.398146 24750 tablet_bootstrap.cc:492] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:06.398190 24555 raft_consensus.cc:2468] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:06.399591 24670 leader_election.cc:304] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:06.402760 24775 raft_consensus.cc:2804] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.403468 24775 raft_consensus.cc:697] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:06.404438 24775 consensus_queue.cc:237] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.406596 24750 tablet_bootstrap.cc:654] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.411023 24782 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.404210 24453 catalog_manager.cc:5697] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 0 to 1, leader changed from <none> to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.429894 24756 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:06.439263 24451 catalog_manager.cc:5697] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 0 to 1, leader changed from <none> to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.449798 24750 tablet_bootstrap.cc:492] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:06.450409 24750 ts_tablet_manager.cc:1403] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.021s
I20260709 12:16:06.453126 24750 raft_consensus.cc:359] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.453857 24750 raft_consensus.cc:385] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.454180 24750 raft_consensus.cc:740] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.454959 24750 consensus_queue.cc:260] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.457096 24750 ts_tablet_manager.cc:1434] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:06.458022 24750 tablet_bootstrap.cc:492] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574: Bootstrap starting.
I20260709 12:16:06.457856 24758 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.461607 24633 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.464262 24750 tablet_bootstrap.cc:654] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.470228 24775 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.473637 24758 consensus_queue.cc:1048] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.493637 24758 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:06.508839 24750 tablet_bootstrap.cc:492] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574: No bootstrap required, opened a new log
I20260709 12:16:06.509493 24750 ts_tablet_manager.cc:1403] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.016s
I20260709 12:16:06.520917 24795 consensus_queue.cc:1048] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:06.532316 24750 raft_consensus.cc:359] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.534870 24750 raft_consensus.cc:385] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.535596 24750 raft_consensus.cc:740] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.537391 24781 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.539819 24750 consensus_queue.cc:260] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.543838 24750 ts_tablet_manager.cc:1434] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.001s
I20260709 12:16:06.628019 24775 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:06.641049 24771 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:06.764398 24781 raft_consensus.cc:493] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.764961 24781 raft_consensus.cc:515] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.767153 24781 leader_election.cc:290] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:06.768334 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105" is_pre_election: true
I20260709 12:16:06.768391 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:06.769287 24555 raft_consensus.cc:2393] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceb6ff4f382f459d9b6b29417d6ff574 in current term 1: Already voted for candidate 750cf88774294edb98402c769337d105 in this term.
I20260709 12:16:06.770709 24596 leader_election.cc:304] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb6ff4f382f459d9b6b29417d6ff574; no voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.771512 24781 raft_consensus.cc:3060] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.776569 24781 raft_consensus.cc:2749] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.797358 24771 raft_consensus.cc:493] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.797930 24771 raft_consensus.cc:515] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.799799 24771 leader_election.cc:290] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.800882 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c146994b1d8431bad2e830d387693fe" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:06.801245 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c146994b1d8431bad2e830d387693fe" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" is_pre_election: true
I20260709 12:16:06.801579 24555 raft_consensus.cc:2468] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
I20260709 12:16:06.801878 24633 raft_consensus.cc:2468] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 0.
I20260709 12:16:06.803005 24670 leader_election.cc:304] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:06.803881 24771 raft_consensus.cc:2804] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:06.804209 24771 raft_consensus.cc:493] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.804507 24771 raft_consensus.cc:3060] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.809453 24771 raft_consensus.cc:515] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.811147 24771 leader_election.cc:290] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:06.812062 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c146994b1d8431bad2e830d387693fe" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:06.812469 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c146994b1d8431bad2e830d387693fe" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:06.812711 24555 raft_consensus.cc:3060] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.813030 24633 raft_consensus.cc:3060] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.816545 24781 raft_consensus.cc:493] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.817613 24781 raft_consensus.cc:515] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.819797 24555 raft_consensus.cc:2468] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:06.820560 24556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1c10cabe1f349938525b635ca28c4f3" is_pre_election: true
I20260709 12:16:06.821067 24670 leader_election.cc:304] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:06.821398 24556 raft_consensus.cc:2393] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ceb6ff4f382f459d9b6b29417d6ff574 in current term 1: Already voted for candidate 750cf88774294edb98402c769337d105 in this term.
I20260709 12:16:06.821965 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750cf88774294edb98402c769337d105" is_pre_election: true
I20260709 12:16:06.822088 24771 raft_consensus.cc:2804] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.822919 24771 raft_consensus.cc:697] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:06.823777 24598 leader_election.cc:304] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceb6ff4f382f459d9b6b29417d6ff574; no voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.822978 24633 raft_consensus.cc:2468] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 1.
I20260709 12:16:06.824479 24771 consensus_queue.cc:237] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:06.825345 24758 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.820660 24781 leader_election.cc:290] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:06.832772 24758 raft_consensus.cc:2749] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.834077 24450 catalog_manager.cc:5697] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 0 to 1, leader changed from <none> to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.868809 24771 consensus_queue.cc:1048] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.873783 24807 consensus_queue.cc:1048] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:06.884248 24807 consensus_queue.cc:1048] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.907991 24775 consensus_queue.cc:1048] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:06.943560 24706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf"
dest_uuid: "750cf88774294edb98402c769337d105"
mode: GRACEFUL
new_leader_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:33454
I20260709 12:16:06.944341 24706 raft_consensus.cc:606] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Received request to transfer leadership to TS b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.950038 24706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5"
dest_uuid: "750cf88774294edb98402c769337d105"
mode: GRACEFUL
new_leader_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:33454
I20260709 12:16:06.950598 24706 raft_consensus.cc:606] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Received request to transfer leadership to TS b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.967180 24632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:33706
I20260709 12:16:06.967875 24632 raft_consensus.cc:606] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Received request to transfer leadership to TS b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.973835 24632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:33706
I20260709 12:16:06.974395 24632 raft_consensus.cc:606] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Received request to transfer leadership to TS b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:06.983565 24706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a"
dest_uuid: "750cf88774294edb98402c769337d105"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33454
I20260709 12:16:06.984227 24706 raft_consensus.cc:606] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:06.989750 24706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca"
dest_uuid: "750cf88774294edb98402c769337d105"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33454
I20260709 12:16:06.990321 24706 raft_consensus.cc:606] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:06.995755 24706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d61a903607af4301a7a84013e37809b0"
dest_uuid: "750cf88774294edb98402c769337d105"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33454
I20260709 12:16:06.996284 24706 raft_consensus.cc:606] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:07.048237 24775 raft_consensus.cc:993] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: : Instructing follower b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.048849 24807 raft_consensus.cc:1081] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Signalling peer b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.050619 24556 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:53826
I20260709 12:16:07.051195 24556 raft_consensus.cc:493] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:07.051496 24556 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.056380 24556 raft_consensus.cc:515] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:07.058203 24556 leader_election.cc:290] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Requested vote from peers 750cf88774294edb98402c769337d105 (127.23.217.67:37701), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:07.058995 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:07.059520 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:07.059719 24706 raft_consensus.cc:3055] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:07.060220 24632 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.060213 24706 raft_consensus.cc:740] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:07.061421 24706 consensus_queue.cc:260] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:07.062880 24706 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.065511 24632 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.066557 24521 leader_election.cc:304] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:07.067302 24818 raft_consensus.cc:2804] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:07.068478 24818 raft_consensus.cc:697] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Leader. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:07.069309 24818 consensus_queue.cc:237] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:07.070324 24706 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.077111 24452 catalog_manager.cc:5697] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 reported cstate change: term changed from 1 to 2, leader changed from 750cf88774294edb98402c769337d105 (127.23.217.67) to b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "b1c10cabe1f349938525b635ca28c4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.085294 24807 raft_consensus.cc:993] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:07.085790 24807 raft_consensus.cc:1081] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:07.087677 24632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33686
I20260709 12:16:07.088263 24632 raft_consensus.cc:493] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:07.088653 24632 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.093423 24775 raft_consensus.cc:993] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105: : Instructing follower b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.093863 24775 raft_consensus.cc:1081] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Signalling peer b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.095716 24556 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:53826
I20260709 12:16:07.096278 24556 raft_consensus.cc:493] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:07.096666 24556 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.096889 24632 raft_consensus.cc:515] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.100108 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:07.100174 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:07.100828 24632 leader_election.cc:290] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:07.101081 24555 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.102512 24706 raft_consensus.cc:3055] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:07.103013 24706 raft_consensus.cc:740] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:07.103833 24556 raft_consensus.cc:515] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.104092 24706 consensus_queue.cc:260] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.105319 24706 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.106792 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:07.107398 24632 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.109241 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:07.108151 24556 leader_election.cc:290] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Requested vote from peers ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:07.109880 24707 raft_consensus.cc:3055] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:07.110229 24707 raft_consensus.cc:740] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:07.111135 24707 consensus_queue.cc:260] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.112341 24707 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.112536 24706 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 2.
I20260709 12:16:07.113869 24598 leader_election.cc:304] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:07.114817 24795 raft_consensus.cc:2804] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:07.115310 24795 raft_consensus.cc:697] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:07.115674 24555 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 2.
I20260709 12:16:07.116128 24795 consensus_queue.cc:237] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.116614 24632 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.118618 24521 leader_election.cc:304] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:07.119668 24818 raft_consensus.cc:2804] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:07.120148 24818 raft_consensus.cc:697] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Leader. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:07.120997 24818 consensus_queue.cc:237] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.123705 24707 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.126020 24451 catalog_manager.cc:5697] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 1 to 2, leader changed from 750cf88774294edb98402c769337d105 (127.23.217.67) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.129942 24452 catalog_manager.cc:5697] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 reported cstate change: term changed from 1 to 2, leader changed from 750cf88774294edb98402c769337d105 (127.23.217.67) to b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "b1c10cabe1f349938525b635ca28c4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.278882 24807 raft_consensus.cc:993] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:07.279495 24807 raft_consensus.cc:1081] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:07.281623 24632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33686
I20260709 12:16:07.282294 24632 raft_consensus.cc:493] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:07.282749 24632 raft_consensus.cc:3060] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.290184 24632 raft_consensus.cc:515] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.292863 24632 leader_election.cc:290] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:07.294209 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:07.294209 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b20d8cc118049678c94d8d489e3caca" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:07.295143 24555 raft_consensus.cc:3060] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.295238 24707 raft_consensus.cc:3055] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:07.295683 24707 raft_consensus.cc:740] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:07.296664 24707 consensus_queue.cc:260] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.298017 24707 raft_consensus.cc:3060] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.303537 24555 raft_consensus.cc:2468] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 2.
I20260709 12:16:07.305433 24596 leader_election.cc:304] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:07.306476 24795 raft_consensus.cc:2804] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:07.307065 24795 raft_consensus.cc:697] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:07.307269 24707 raft_consensus.cc:2468] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 2.
I20260709 12:16:07.308059 24795 consensus_queue.cc:237] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.316666 24452 catalog_manager.cc:5697] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 1 to 2, leader changed from 750cf88774294edb98402c769337d105 (127.23.217.67) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.330667 24807 consensus_queue.cc:1048] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:07.346216 24756 consensus_queue.cc:1048] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:07.444147 24781 raft_consensus.cc:993] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.444662 24795 raft_consensus.cc:1081] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Signalling peer b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.446252 24556 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:53834
I20260709 12:16:07.446851 24556 raft_consensus.cc:493] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:07.447175 24556 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.452734 24556 raft_consensus.cc:515] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.455399 24556 leader_election.cc:290] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Requested vote from peers ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:07.456319 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:07.457060 24632 raft_consensus.cc:3055] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:07.457373 24632 raft_consensus.cc:740] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:07.458209 24632 consensus_queue.cc:260] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.459264 24632 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.464663 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:07.465195 24707 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.467973 24632 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.470911 24521 leader_election.cc:304] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:07.472033 24818 raft_consensus.cc:2804] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:07.472471 24818 raft_consensus.cc:697] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Leader. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:07.473220 24818 consensus_queue.cc:237] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.481593 24707 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.482589 24451 catalog_manager.cc:5697] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 reported cstate change: term changed from 1 to 2, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "b1c10cabe1f349938525b635ca28c4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.497068 24781 raft_consensus.cc:993] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.497475 24795 raft_consensus.cc:1081] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Signalling peer b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:07.499207 24556 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:53834
I20260709 12:16:07.499886 24556 raft_consensus.cc:493] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:07.500309 24556 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.505793 24556 raft_consensus.cc:515] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.507694 24556 leader_election.cc:290] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Requested vote from peers ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:07.508435 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:07.508664 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:07.509093 24632 raft_consensus.cc:3055] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:07.509153 24707 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.509568 24632 raft_consensus.cc:740] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:07.510362 24632 consensus_queue.cc:260] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.511417 24632 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.516378 24707 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.516803 24632 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 2.
I20260709 12:16:07.517589 24523 leader_election.cc:304] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:07.518926 24833 raft_consensus.cc:2804] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:07.519439 24833 raft_consensus.cc:697] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Leader. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:07.520306 24833 consensus_queue.cc:237] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.520820 24707 raft_consensus.cc:1275] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.522545 24771 raft_consensus.cc:993] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:07.522279 24818 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.522982 24807 raft_consensus.cc:1081] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:07.524585 24632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d61a903607af4301a7a84013e37809b0"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33686
I20260709 12:16:07.525043 24632 raft_consensus.cc:493] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:07.525372 24632 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.531023 24632 raft_consensus.cc:515] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.533581 24632 leader_election.cc:290] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:07.534128 24556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:07.534793 24556 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.537503 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:07.538144 24706 raft_consensus.cc:3055] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:07.538527 24706 raft_consensus.cc:740] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:07.539399 24706 consensus_queue.cc:260] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.539479 24452 catalog_manager.cc:5697] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 reported cstate change: term changed from 1 to 2, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "b1c10cabe1f349938525b635ca28c4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.542023 24632 raft_consensus.cc:1275] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.543336 24556 raft_consensus.cc:2468] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 2.
I20260709 12:16:07.543574 24818 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.544646 24596 leader_election.cc:304] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:07.540895 24706 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:07.545646 24795 raft_consensus.cc:2804] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:07.546154 24795 raft_consensus.cc:697] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:07.547055 24795 consensus_queue.cc:237] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:07.554687 24706 raft_consensus.cc:2468] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 2.
I20260709 12:16:07.563760 24452 catalog_manager.cc:5697] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 1 to 2, leader changed from 750cf88774294edb98402c769337d105 (127.23.217.67) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.584599 24706 raft_consensus.cc:1275] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.585916 24818 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.594630 24632 raft_consensus.cc:1275] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.597445 24838 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.732016 24556 raft_consensus.cc:1275] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.733429 24839 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.738442 24555 raft_consensus.cc:1275] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.741110 24781 consensus_queue.cc:1048] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:16:07.744146 24706 raft_consensus.cc:1275] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.745846 24781 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.755512 24707 raft_consensus.cc:1275] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.757051 24795 consensus_queue.cc:1048] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.930364 24632 raft_consensus.cc:1275] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.931660 24838 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:07.940305 24707 raft_consensus.cc:1275] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:07.941907 24818 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:08.033955 24707 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:08.035395 24818 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:08.044312 24632 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:08.046386 24842 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:08.106233 24706 raft_consensus.cc:1275] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:08.107452 24795 consensus_queue.cc:1048] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:08.115969 24556 raft_consensus.cc:1275] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:08.118230 24795 consensus_queue.cc:1048] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:10.033579 24421 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:16:10.034618 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:10.051769 24632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:33716
I20260709 12:16:10.052295 24632 raft_consensus.cc:606] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Received request to transfer leadership to TS 750cf88774294edb98402c769337d105
I20260709 12:16:10.064121 24555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
mode: GRACEFUL
new_leader_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:53840
I20260709 12:16:10.064615 24555 raft_consensus.cc:606] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Received request to transfer leadership to TS 750cf88774294edb98402c769337d105
I20260709 12:16:10.065750 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:10.066201 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:10.091661 24838 raft_consensus.cc:993] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3: : Instructing follower 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:10.092164 24856 raft_consensus.cc:1081] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Signalling peer 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:10.093863 24707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:33446
I20260709 12:16:10.094413 24707 raft_consensus.cc:493] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:10.094825 24707 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:10.099845 24707 raft_consensus.cc:515] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:10.101425 24707 leader_election.cc:290] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 3 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:10.102383 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:10.102733 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:10.103075 24555 raft_consensus.cc:3055] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:10.103260 24632 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:10.103466 24555 raft_consensus.cc:740] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:10.104314 24555 consensus_queue.cc:260] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:10.105577 24555 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:10.110189 24632 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 3.
I20260709 12:16:10.111542 24670 leader_election.cc:304] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:10.111963 24555 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 3.
I20260709 12:16:10.112481 24771 raft_consensus.cc:2804] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:10.113987 24771 raft_consensus.cc:697] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:10.115026 24771 consensus_queue.cc:237] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:10.125213 24448 catalog_manager.cc:5697] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 2 to 3, leader changed from b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65) to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:10.202947 24863 raft_consensus.cc:993] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:10.203531 24839 raft_consensus.cc:1081] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Signalling peer 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:10.205219 24707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a"
dest_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:33436
I20260709 12:16:10.205725 24707 raft_consensus.cc:493] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:10.206086 24707 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:10.211190 24707 raft_consensus.cc:515] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:10.212754 24707 leader_election.cc:290] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 3 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:10.213651 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:10.213963 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:10.214314 24555 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:10.214512 24632 raft_consensus.cc:3055] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:10.214865 24632 raft_consensus.cc:740] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:10.215682 24632 consensus_queue.cc:260] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:10.216871 24632 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:10.221118 24555 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 3.
I20260709 12:16:10.222388 24670 leader_election.cc:304] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:10.222900 24632 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 3.
I20260709 12:16:10.223410 24771 raft_consensus.cc:2804] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:10.224021 24771 raft_consensus.cc:697] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:10.224896 24771 consensus_queue.cc:237] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:10.232496 24448 catalog_manager.cc:5697] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 2 to 3, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:10.496922 24555 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:10.498201 24771 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:10.510325 24632 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:10.511618 24771 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:10.700913 24556 raft_consensus.cc:1275] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:10.702174 24771 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:10.711934 24632 raft_consensus.cc:1275] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:10.713330 24771 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:11.067481 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:11.073096 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:11.073530 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:12.074910 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:12.080996 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:12.081504 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:13.082715 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:13.088315 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:13.088740 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:14.089919 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:14.095011 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:14.095453 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:15.096457 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:15.101109 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:15.101521 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:16.102596 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:16.107550 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:16.107909 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:17.108825 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:17.114480 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:17.114985 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:18.116039 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:18.121189 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:18.121704 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:19.123016 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:19.128680 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:19.129139 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:20.156833 24421 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:16:20.171303 24555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:44918
I20260709 12:16:20.172008 24555 raft_consensus.cc:606] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:20.177165 24555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:44918
I20260709 12:16:20.177623 24555 raft_consensus.cc:606] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:20.182345 24555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:44918
I20260709 12:16:20.182917 24555 raft_consensus.cc:606] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:20.187742 24706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "750cf88774294edb98402c769337d105"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33454
I20260709 12:16:20.188266 24706 raft_consensus.cc:606] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:20.195288 24706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a"
dest_uuid: "750cf88774294edb98402c769337d105"
mode: GRACEFUL
new_leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33454
I20260709 12:16:20.195915 24706 raft_consensus.cc:606] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Received request to transfer leadership to TS ceb6ff4f382f459d9b6b29417d6ff574
I20260709 12:16:20.208716 24876 raft_consensus.cc:993] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.209167 24876 raft_consensus.cc:1081] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.210626 24633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33696
I20260709 12:16:20.211171 24633 raft_consensus.cc:493] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.211585 24633 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.217237 24633 raft_consensus.cc:515] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.218947 24633 leader_election.cc:290] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 3 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:20.219884 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:20.220314 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:20.220808 24555 raft_consensus.cc:3055] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:20.220927 24707 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.221114 24555 raft_consensus.cc:740] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:20.221907 24555 consensus_queue.cc:260] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.222968 24555 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.225915 24707 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 3.
I20260709 12:16:20.227119 24598 leader_election.cc:304] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:20.227936 24839 raft_consensus.cc:2804] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:20.228219 24555 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 3.
I20260709 12:16:20.229008 24839 raft_consensus.cc:697] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:20.230003 24839 consensus_queue.cc:237] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.237414 24448 catalog_manager.cc:5697] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 2 to 3, leader changed from b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.366178 24878 raft_consensus.cc:993] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.366716 24878 raft_consensus.cc:1081] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.368242 24633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33686
I20260709 12:16:20.368794 24633 raft_consensus.cc:493] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.369110 24633 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:20.373538 24633 raft_consensus.cc:515] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.375183 24633 leader_election.cc:290] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 4 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:20.376206 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:20.376324 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:20.376895 24555 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:20.376987 24707 raft_consensus.cc:3055] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:20.377379 24707 raft_consensus.cc:740] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:20.378072 24707 consensus_queue.cc:260] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.379036 24707 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:20.381908 24555 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 4.
I20260709 12:16:20.382987 24596 leader_election.cc:304] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:20.383828 24839 raft_consensus.cc:2804] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:20.383898 24707 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 4.
I20260709 12:16:20.384276 24839 raft_consensus.cc:697] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:20.385119 24839 consensus_queue.cc:237] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.391897 24448 catalog_manager.cc:5697] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 3 to 4, leader changed from 750cf88774294edb98402c769337d105 (127.23.217.67) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 4 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.414685 24889 raft_consensus.cc:993] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.415132 24889 raft_consensus.cc:1081] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.416500 24633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33686
I20260709 12:16:20.416945 24633 raft_consensus.cc:493] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.417228 24633 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:20.421658 24633 raft_consensus.cc:515] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.423430 24633 leader_election.cc:290] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 4 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:20.424378 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:20.424890 24555 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:20.425155 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5be7a7bc10504a6586d316a44947410a" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:20.425791 24707 raft_consensus.cc:3055] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:20.426121 24707 raft_consensus.cc:740] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:20.427256 24707 consensus_queue.cc:260] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.428516 24707 raft_consensus.cc:3060] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:20.429979 24555 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 4.
I20260709 12:16:20.430198 24882 raft_consensus.cc:993] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.430797 24882 raft_consensus.cc:1081] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.430956 24596 leader_election.cc:304] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:20.431696 24839 raft_consensus.cc:2804] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:20.432343 24839 raft_consensus.cc:697] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:20.432704 24633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33696
I20260709 12:16:20.433262 24633 raft_consensus.cc:493] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.433679 24633 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.433204 24839 consensus_queue.cc:237] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.434825 24707 raft_consensus.cc:2468] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 4.
I20260709 12:16:20.439262 24633 raft_consensus.cc:515] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:20.440830 24633 leader_election.cc:290] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 3 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:20.441480 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:20.442050 24555 raft_consensus.cc:3055] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:20.442428 24555 raft_consensus.cc:740] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:20.442534 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:20.441800 24448 catalog_manager.cc:5697] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 3 to 4, leader changed from 750cf88774294edb98402c769337d105 (127.23.217.67) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 4 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.443230 24555 consensus_queue.cc:260] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:20.444100 24707 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.444444 24555 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.449954 24555 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 3.
I20260709 12:16:20.449954 24707 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 3.
I20260709 12:16:20.451383 24598 leader_election.cc:304] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:20.452462 24839 raft_consensus.cc:2804] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:20.453094 24839 raft_consensus.cc:697] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:20.453917 24839 consensus_queue.cc:237] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:20.460511 24448 catalog_manager.cc:5697] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 2 to 3, leader changed from b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:20.468923 24882 raft_consensus.cc:993] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3: : Instructing follower ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.469401 24876 raft_consensus.cc:1081] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Signalling peer ceb6ff4f382f459d9b6b29417d6ff574 to start an election
I20260709 12:16:20.470832 24633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
 from {username='slave'} at 127.0.0.1:33696
I20260709 12:16:20.471261 24633 raft_consensus.cc:493] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.471513 24633 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.475304 24633 raft_consensus.cc:515] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.476807 24633 leader_election.cc:290] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 3 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:20.477726 24555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:20.477746 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:20.478531 24555 raft_consensus.cc:3055] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:20.478549 24707 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.478991 24555 raft_consensus.cc:740] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:20.479724 24555 consensus_queue.cc:260] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.480755 24555 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:20.482869 24707 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 3.
I20260709 12:16:20.483858 24598 leader_election.cc:304] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:20.484529 24839 raft_consensus.cc:2804] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:20.485001 24839 raft_consensus.cc:697] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Becoming Leader. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:20.485824 24839 consensus_queue.cc:237] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:20.486785 24555 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb6ff4f382f459d9b6b29417d6ff574 in term 3.
I20260709 12:16:20.493764 24448 catalog_manager.cc:5697] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 reported cstate change: term changed from 2 to 3, leader changed from b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65) to ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.768956 24555 raft_consensus.cc:1275] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:20.770542 24839 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:20.782366 24707 raft_consensus.cc:1275] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:20.784010 24896 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:20.945134 24555 raft_consensus.cc:1275] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:20.946341 24839 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:20.954825 24707 raft_consensus.cc:1275] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:20.956215 24883 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:20.970310 24555 raft_consensus.cc:1275] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:20.971561 24883 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:20.982817 24707 raft_consensus.cc:1275] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:20.983132 24706 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:20.984550 24883 consensus_queue.cc:1048] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:20.984787 24896 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:20.997191 24556 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:20.999017 24883 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:21.065619 24706 raft_consensus.cc:1275] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:21.065626 24556 raft_consensus.cc:1275] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Refusing update from remote peer ceb6ff4f382f459d9b6b29417d6ff574: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:21.067263 24839 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:21.068086 24896 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:23.237475 24421 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:16:23.238348 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:23.245361 24633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:33716
I20260709 12:16:23.246002 24633 raft_consensus.cc:606] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Received request to transfer leadership to TS 750cf88774294edb98402c769337d105
I20260709 12:16:23.248901 24633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:33716
I20260709 12:16:23.249543 24633 raft_consensus.cc:606] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 LEADER]: Received request to transfer leadership to TS b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:23.251892 24633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:33716
I20260709 12:16:23.252269 24633 raft_consensus.cc:606] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Received request to transfer leadership to TS b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:23.254283 24633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:33716
I20260709 12:16:23.254691 24633 raft_consensus.cc:606] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Received request to transfer leadership to TS 750cf88774294edb98402c769337d105
I20260709 12:16:23.256704 24633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d61a903607af4301a7a84013e37809b0"
dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
mode: GRACEFUL
new_leader_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:33716
I20260709 12:16:23.257036 24633 raft_consensus.cc:606] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Received request to transfer leadership to TS b1c10cabe1f349938525b635ca28c4f3
I20260709 12:16:23.257889 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:23.258262 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:23.401834 24896 raft_consensus.cc:993] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:23.402472 24911 raft_consensus.cc:1081] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Signalling peer 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:23.404127 24707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf"
dest_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:33436
I20260709 12:16:23.404650 24707 raft_consensus.cc:493] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:23.404960 24707 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.410156 24707 raft_consensus.cc:515] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:23.412075 24707 leader_election.cc:290] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 4 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:23.413437 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:23.413463 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af595ba369a4e15a50274c854c116bf" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:23.414108 24633 raft_consensus.cc:3055] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:23.414278 24558 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.414369 24633 raft_consensus.cc:740] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:23.415262 24633 consensus_queue.cc:260] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:23.416505 24633 raft_consensus.cc:3060] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.420812 24911 raft_consensus.cc:993] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:23.421839 24911 raft_consensus.cc:1081] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Signalling peer b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:23.422995 24633 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 4.
I20260709 12:16:23.423743 24555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:53834
I20260709 12:16:23.424301 24555 raft_consensus.cc:493] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:23.424679 24555 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.424670 24558 raft_consensus.cc:2468] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 4.
I20260709 12:16:23.425145 24670 leader_election.cc:304] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:23.426246 24910 raft_consensus.cc:2804] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:23.427378 24910 raft_consensus.cc:697] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 4 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:23.428419 24910 consensus_queue.cc:237] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } }
I20260709 12:16:23.431578 24555 raft_consensus.cc:515] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.435236 24555 leader_election.cc:290] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 4 election: Requested vote from peers ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:23.435302 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:23.435652 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912720e8d5b644e4bceda48a449b9b8d" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:23.435938 24633 raft_consensus.cc:3055] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:23.436151 24707 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.436250 24633 raft_consensus.cc:740] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:23.437029 24633 consensus_queue.cc:260] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.438122 24633 raft_consensus.cc:3060] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.438661 24452 catalog_manager.cc:5697] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 3 to 4, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:23.442459 24707 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 4.
I20260709 12:16:23.444041 24523 leader_election.cc:304] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:23.444342 24633 raft_consensus.cc:2468] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 4.
I20260709 12:16:23.446442 24917 raft_consensus.cc:2804] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:23.447929 24917 raft_consensus.cc:697] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 4 LEADER]: Becoming Leader. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:23.448917 24917 consensus_queue.cc:237] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.457235 24452 catalog_manager.cc:5697] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 reported cstate change: term changed from 3 to 4, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65). New cstate: current_term: 4 leader_uuid: "b1c10cabe1f349938525b635ca28c4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:23.467697 24896 raft_consensus.cc:993] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:23.468216 24911 raft_consensus.cc:993] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:23.468408 24913 raft_consensus.cc:1081] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 LEADER]: Signalling peer b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:23.468705 24911 raft_consensus.cc:1081] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Signalling peer b1c10cabe1f349938525b635ca28c4f3 to start an election
I20260709 12:16:23.470093 24558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:53834
I20260709 12:16:23.470556 24555 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d61a903607af4301a7a84013e37809b0"
dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
 from {username='slave'} at 127.0.0.1:53834
I20260709 12:16:23.470731 24558 raft_consensus.cc:493] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:23.471052 24555 raft_consensus.cc:493] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:23.471175 24558 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:23.471378 24555 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:23.475414 24555 raft_consensus.cc:515] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.476819 24555 leader_election.cc:290] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 3 election: Requested vote from peers ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:23.477450 24633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:23.476559 24558 raft_consensus.cc:515] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.477888 24707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d61a903607af4301a7a84013e37809b0" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:23.478410 24707 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:23.477998 24633 raft_consensus.cc:3055] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:23.478892 24633 raft_consensus.cc:740] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:23.479710 24558 leader_election.cc:290] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 5 election: Requested vote from peers ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233), 750cf88774294edb98402c769337d105 (127.23.217.67:37701)
I20260709 12:16:23.479780 24633 consensus_queue.cc:260] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.481154 24633 raft_consensus.cc:3060] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:23.484323 24706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "750cf88774294edb98402c769337d105"
I20260709 12:16:23.484870 24706 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:23.485232 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5b4d278c17437ab3c075b769910253" candidate_uuid: "b1c10cabe1f349938525b635ca28c4f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:23.485745 24632 raft_consensus.cc:3055] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:23.486006 24632 raft_consensus.cc:740] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:23.486888 24632 consensus_queue.cc:260] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.488132 24632 raft_consensus.cc:3060] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:23.488417 24633 raft_consensus.cc:2468] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 3.
I20260709 12:16:23.489465 24521 leader_election.cc:304] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1c10cabe1f349938525b635ca28c4f3, ceb6ff4f382f459d9b6b29417d6ff574; no voters: 
I20260709 12:16:23.492077 24707 raft_consensus.cc:2468] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 3.
I20260709 12:16:23.492462 24706 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 5.
I20260709 12:16:23.493188 24917 raft_consensus.cc:2804] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:23.493955 24917 raft_consensus.cc:697] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 3 LEADER]: Becoming Leader. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:23.494421 24523 leader_election.cc:304] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:23.494840 24632 raft_consensus.cc:2468] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1c10cabe1f349938525b635ca28c4f3 in term 5.
I20260709 12:16:23.495298 24919 raft_consensus.cc:2804] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:23.494951 24917 consensus_queue.cc:237] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.496699 24919 raft_consensus.cc:697] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 5 LEADER]: Becoming Leader. State: Replica: b1c10cabe1f349938525b635ca28c4f3, State: Running, Role: LEADER
I20260709 12:16:23.497562 24919 consensus_queue.cc:237] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.503291 24452 catalog_manager.cc:5697] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 reported cstate change: term changed from 2 to 3, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65). New cstate: current_term: 3 leader_uuid: "b1c10cabe1f349938525b635ca28c4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:23.514635 24452 catalog_manager.cc:5697] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 reported cstate change: term changed from 4 to 5, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65). New cstate: current_term: 5 leader_uuid: "b1c10cabe1f349938525b635ca28c4f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:23.563030 24911 raft_consensus.cc:993] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574: : Instructing follower 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:23.563516 24911 raft_consensus.cc:1081] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Signalling peer 750cf88774294edb98402c769337d105 to start an election
I20260709 12:16:23.565016 24706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5"
dest_uuid: "750cf88774294edb98402c769337d105"
 from {username='slave'} at 127.0.0.1:33436
I20260709 12:16:23.565515 24706 raft_consensus.cc:493] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:23.565770 24706 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.570369 24706 raft_consensus.cc:515] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.571825 24706 leader_election.cc:290] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 4 election: Requested vote from peers b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443), ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233)
I20260709 12:16:23.572873 24632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ceb6ff4f382f459d9b6b29417d6ff574"
I20260709 12:16:23.572782 24558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91da17b78bd4f188511e5ae1ee51da5" candidate_uuid: "750cf88774294edb98402c769337d105" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b1c10cabe1f349938525b635ca28c4f3"
I20260709 12:16:23.573488 24632 raft_consensus.cc:3055] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:23.573570 24558 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.573879 24632 raft_consensus.cc:740] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ceb6ff4f382f459d9b6b29417d6ff574, State: Running, Role: LEADER
I20260709 12:16:23.574663 24632 consensus_queue.cc:260] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.575855 24632 raft_consensus.cc:3060] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:23.580667 24558 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 4.
I20260709 12:16:23.581954 24670 leader_election.cc:304] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750cf88774294edb98402c769337d105, b1c10cabe1f349938525b635ca28c4f3; no voters: 
I20260709 12:16:23.582043 24632 raft_consensus.cc:2468] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 750cf88774294edb98402c769337d105 in term 4.
I20260709 12:16:23.582896 24910 raft_consensus.cc:2804] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:23.583482 24910 raft_consensus.cc:697] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 4 LEADER]: Becoming Leader. State: Replica: 750cf88774294edb98402c769337d105, State: Running, Role: LEADER
I20260709 12:16:23.584430 24910 consensus_queue.cc:237] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } }
I20260709 12:16:23.592032 24452 catalog_manager.cc:5697] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 reported cstate change: term changed from 3 to 4, leader changed from ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66) to 750cf88774294edb98402c769337d105 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "750cf88774294edb98402c769337d105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:23.807590 24558 raft_consensus.cc:1275] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:23.808810 24910 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:23.819470 24632 raft_consensus.cc:1275] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:23.821388 24930 consensus_queue.cc:1048] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:23.903807 24632 raft_consensus.cc:1275] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:23.905264 24919 consensus_queue.cc:1048] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:23.908334 24633 raft_consensus.cc:1275] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:23.909958 24917 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:23.918766 24706 raft_consensus.cc:1275] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:23.922031 24924 consensus_queue.cc:1048] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:23.928262 24707 raft_consensus.cc:1275] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:23.933838 24919 consensus_queue.cc:1048] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:23.948426 24706 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 5 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:23.949465 24919 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750cf88774294edb98402c769337d105" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:23.958282 24632 raft_consensus.cc:1275] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 5 FOLLOWER]: Refusing update from remote peer b1c10cabe1f349938525b635ca28c4f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:23.960170 24919 consensus_queue.cc:1048] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:24.027700 24632 raft_consensus.cc:1275] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:24.029188 24930 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb6ff4f382f459d9b6b29417d6ff574" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:24.038525 24558 raft_consensus.cc:1275] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Refusing update from remote peer 750cf88774294edb98402c769337d105: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:24.040326 24910 consensus_queue.cc:1048] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1c10cabe1f349938525b635ca28c4f3" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:24.259625 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:24.265080 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:24.265576 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:25.266916 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:25.272332 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:25.272830 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:26.273972 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:26.279047 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:26.279632 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:27.280700 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:27.285920 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:27.286307 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:28.287290 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:28.292275 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:28.292665 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:29.293817 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.299032 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:29.299484 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:30.300554 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:30.305804 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:30.306198 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:31.307623 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:31.312780 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:31.313262 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:32.314397 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:32.320549 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:32.320990 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:33.351568 24421 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:16:33.352535 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
I20260709 12:16:33.391134 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:33.392006 24421 tablet_replica.cc:333] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.392832 24421 raft_consensus.cc:2243] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.393721 24421 raft_consensus.cc:2272] T 5be7a7bc10504a6586d316a44947410a P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.396122 24421 tablet_replica.cc:333] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.396634 24421 raft_consensus.cc:2243] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.397433 24421 raft_consensus.cc:2272] T 912720e8d5b644e4bceda48a449b9b8d P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.399153 24421 tablet_replica.cc:333] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.399721 24421 raft_consensus.cc:2243] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.400143 24421 raft_consensus.cc:2272] T 1b20d8cc118049678c94d8d489e3caca P b1c10cabe1f349938525b635ca28c4f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.401652 24421 tablet_replica.cc:333] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.402153 24421 raft_consensus.cc:2243] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.403024 24421 raft_consensus.cc:2272] T d61a903607af4301a7a84013e37809b0 P b1c10cabe1f349938525b635ca28c4f3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.404611 24421 tablet_replica.cc:333] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.405085 24421 raft_consensus.cc:2243] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.405784 24421 raft_consensus.cc:2272] T 6e5b4d278c17437ab3c075b769910253 P b1c10cabe1f349938525b635ca28c4f3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.407630 24421 tablet_replica.cc:333] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.408083 24421 raft_consensus.cc:2243] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.408458 24421 raft_consensus.cc:2272] T 0c146994b1d8431bad2e830d387693fe P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.410002 24421 tablet_replica.cc:333] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.410475 24421 raft_consensus.cc:2243] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.410908 24421 raft_consensus.cc:2272] T 04d91e2c217b4008bc9643cccd37fe93 P b1c10cabe1f349938525b635ca28c4f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.412577 24421 tablet_replica.cc:333] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.413018 24421 raft_consensus.cc:2243] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.413442 24421 raft_consensus.cc:2272] T 3af595ba369a4e15a50274c854c116bf P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.414917 24421 tablet_replica.cc:333] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3: stopping tablet replica
I20260709 12:16:33.415585 24421 raft_consensus.cc:2243] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.416204 24421 raft_consensus.cc:2272] T a91da17b78bd4f188511e5ae1ee51da5 P b1c10cabe1f349938525b635ca28c4f3 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:33.519973 24596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.65:45443: connect: Connection refused (error 111)
I20260709 12:16:33.521652 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
W20260709 12:16:33.524479 24596 consensus_peers.cc:597] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 -> Peer b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443): Couldn't send request to peer b1c10cabe1f349938525b635ca28c4f3. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:45443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:33.541373 24670 consensus_peers.cc:597] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 -> Peer b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443): Couldn't send request to peer b1c10cabe1f349938525b635ca28c4f3. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:45443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.544701 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
W20260709 12:16:33.567080 24596 consensus_peers.cc:597] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 -> Peer b1c10cabe1f349938525b635ca28c4f3 (127.23.217.65:45443): Couldn't send request to peer b1c10cabe1f349938525b635ca28c4f3. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:45443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.583981 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:33.584904 24421 tablet_replica.cc:333] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.585717 24421 raft_consensus.cc:2243] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.586232 24421 raft_consensus.cc:2272] T 0c146994b1d8431bad2e830d387693fe P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.588579 24421 tablet_replica.cc:333] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.589197 24421 raft_consensus.cc:2243] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.589742 24421 raft_consensus.cc:2272] T 3af595ba369a4e15a50274c854c116bf P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.591755 24421 tablet_replica.cc:333] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.592293 24421 raft_consensus.cc:2243] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.592756 24421 raft_consensus.cc:2272] T a91da17b78bd4f188511e5ae1ee51da5 P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.594439 24421 tablet_replica.cc:333] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.595032 24421 raft_consensus.cc:2243] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.595520 24421 raft_consensus.cc:2272] T d61a903607af4301a7a84013e37809b0 P ceb6ff4f382f459d9b6b29417d6ff574 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.597165 24421 tablet_replica.cc:333] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.597663 24421 raft_consensus.cc:2243] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.598587 24421 raft_consensus.cc:2272] T 1b20d8cc118049678c94d8d489e3caca P ceb6ff4f382f459d9b6b29417d6ff574 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.600203 24421 tablet_replica.cc:333] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.600735 24421 raft_consensus.cc:2243] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.601171 24421 raft_consensus.cc:2272] T 6e5b4d278c17437ab3c075b769910253 P ceb6ff4f382f459d9b6b29417d6ff574 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.603112 24421 tablet_replica.cc:333] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.603636 24421 raft_consensus.cc:2243] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.604048 24421 raft_consensus.cc:2272] T 912720e8d5b644e4bceda48a449b9b8d P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.605788 24421 tablet_replica.cc:333] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.606271 24421 raft_consensus.cc:2243] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.607111 24421 raft_consensus.cc:2272] T 5be7a7bc10504a6586d316a44947410a P ceb6ff4f382f459d9b6b29417d6ff574 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.608812 24421 tablet_replica.cc:333] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574: stopping tablet replica
I20260709 12:16:33.609268 24421 raft_consensus.cc:2243] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.610024 24421 raft_consensus.cc:2272] T 04d91e2c217b4008bc9643cccd37fe93 P ceb6ff4f382f459d9b6b29417d6ff574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.641420 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
W20260709 12:16:33.664316 24670 consensus_peers.cc:597] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 -> Peer ceb6ff4f382f459d9b6b29417d6ff574 (127.23.217.66:40233): Couldn't send request to peer ceb6ff4f382f459d9b6b29417d6ff574. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:40233: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.667058 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
I20260709 12:16:33.700767 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:33.701586 24421 tablet_replica.cc:333] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.702190 24421 raft_consensus.cc:2243] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.703022 24421 raft_consensus.cc:2272] T 0c146994b1d8431bad2e830d387693fe P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.704778 24421 tablet_replica.cc:333] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.705291 24421 raft_consensus.cc:2243] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.705713 24421 raft_consensus.cc:2272] T 04d91e2c217b4008bc9643cccd37fe93 P 750cf88774294edb98402c769337d105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.707476 24421 tablet_replica.cc:333] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.707984 24421 raft_consensus.cc:2243] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.708418 24421 raft_consensus.cc:2272] T d61a903607af4301a7a84013e37809b0 P 750cf88774294edb98402c769337d105 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.710016 24421 tablet_replica.cc:333] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.710528 24421 raft_consensus.cc:2243] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.710918 24421 raft_consensus.cc:2272] T 1b20d8cc118049678c94d8d489e3caca P 750cf88774294edb98402c769337d105 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.712460 24421 tablet_replica.cc:333] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.713007 24421 raft_consensus.cc:2243] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.713421 24421 raft_consensus.cc:2272] T 6e5b4d278c17437ab3c075b769910253 P 750cf88774294edb98402c769337d105 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.714998 24421 tablet_replica.cc:333] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.715449 24421 raft_consensus.cc:2243] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.715866 24421 raft_consensus.cc:2272] T 912720e8d5b644e4bceda48a449b9b8d P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.717384 24421 tablet_replica.cc:333] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.717834 24421 raft_consensus.cc:2243] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.718216 24421 raft_consensus.cc:2272] T 5be7a7bc10504a6586d316a44947410a P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.719779 24421 tablet_replica.cc:333] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.720209 24421 raft_consensus.cc:2243] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.720937 24421 raft_consensus.cc:2272] T a91da17b78bd4f188511e5ae1ee51da5 P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.722446 24421 tablet_replica.cc:333] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105: stopping tablet replica
I20260709 12:16:33.722919 24421 raft_consensus.cc:2243] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.723620 24421 raft_consensus.cc:2272] T 3af595ba369a4e15a50274c854c116bf P 750cf88774294edb98402c769337d105 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.752076 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
I20260709 12:16:33.771489 24421 master.cc:562] Master@127.23.217.126:39131 shutting down...
I20260709 12:16:33.911242 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:33.911911 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.912251 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P bde9f1b1bd88476cbac796c31003312f: stopping tablet replica
I20260709 12:16:33.933584 24421 master.cc:584] Master@127.23.217.126:39131 shutdown complete.
750cf88774294edb98402c769337d105  1
b1c10cabe1f349938525b635ca28c4f3  4
ceb6ff4f382f459d9b6b29417d6ff574  4
750cf88774294edb98402c769337d105  3
b1c10cabe1f349938525b635ca28c4f3  3
ceb6ff4f382f459d9b6b29417d6ff574  3
750cf88774294edb98402c769337d105  1
b1c10cabe1f349938525b635ca28c4f3  0
ceb6ff4f382f459d9b6b29417d6ff574  8
750cf88774294edb98402c769337d105  3
b1c10cabe1f349938525b635ca28c4f3  3
ceb6ff4f382f459d9b6b29417d6ff574  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31302 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:16:33.977034 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:39547
I20260709 12:16:33.978171 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:33.984118 24957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:33.984807 24958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:33.987906 24960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:33.988361 24421 server_base.cc:1061] running on GCE node
I20260709 12:16:33.989537 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:33.989761 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:33.989931 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599393989914 us; error 0 us; skew 500 ppm
I20260709 12:16:33.990447 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:33.993396 24421 webserver.cc:533] Webserver started at http://127.23.217.126:37635/ using document root <none> and password file <none>
I20260709 12:16:33.993899 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:33.994102 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:33.994378 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:33.995612 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "99f814c53bc7430ca37581ee70d95fb7"
format_stamp: "Formatted at 2026-07-09 12:16:33 on dist-test-slave-lg8f"
I20260709 12:16:34.000128 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:16:34.003710 24965 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.004601 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:16:34.004928 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "99f814c53bc7430ca37581ee70d95fb7"
format_stamp: "Formatted at 2026-07-09 12:16:33 on dist-test-slave-lg8f"
I20260709 12:16:34.005219 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:34.031432 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:34.032681 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:34.072082 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:39547
I20260709 12:16:34.072257 25016 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:39547 every 8 connection(s)
I20260709 12:16:34.076279 25017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.086607 25017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7: Bootstrap starting.
I20260709 12:16:34.091773 25017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.096407 25017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7: No bootstrap required, opened a new log
I20260709 12:16:34.098820 25017 raft_consensus.cc:359] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f814c53bc7430ca37581ee70d95fb7" member_type: VOTER }
I20260709 12:16:34.099342 25017 raft_consensus.cc:385] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.099574 25017 raft_consensus.cc:740] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f814c53bc7430ca37581ee70d95fb7, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.100214 25017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f814c53bc7430ca37581ee70d95fb7" member_type: VOTER }
I20260709 12:16:34.100710 25017 raft_consensus.cc:399] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:34.100958 25017 raft_consensus.cc:493] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:34.101240 25017 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.106412 25017 raft_consensus.cc:515] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f814c53bc7430ca37581ee70d95fb7" member_type: VOTER }
I20260709 12:16:34.107061 25017 leader_election.cc:304] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99f814c53bc7430ca37581ee70d95fb7; no voters: 
I20260709 12:16:34.108331 25017 leader_election.cc:290] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:34.108886 25020 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.110338 25020 raft_consensus.cc:697] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 99f814c53bc7430ca37581ee70d95fb7, State: Running, Role: LEADER
I20260709 12:16:34.111088 25020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f814c53bc7430ca37581ee70d95fb7" member_type: VOTER }
I20260709 12:16:34.111536 25017 sys_catalog.cc:565] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:34.116092 25021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99f814c53bc7430ca37581ee70d95fb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f814c53bc7430ca37581ee70d95fb7" member_type: VOTER } }
I20260709 12:16:34.116707 25021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:34.117221 25022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99f814c53bc7430ca37581ee70d95fb7. Latest consensus state: current_term: 1 leader_uuid: "99f814c53bc7430ca37581ee70d95fb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f814c53bc7430ca37581ee70d95fb7" member_type: VOTER } }
I20260709 12:16:34.117902 25022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:34.120400 25028 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:34.129534 25028 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:34.130651 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:34.138903 25028 catalog_manager.cc:1383] Generated new cluster ID: 3d9dcee825f840569a730ff06b692a55
I20260709 12:16:34.139194 25028 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:34.153082 25028 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:34.154417 25028 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:34.167488 25028 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7: Generated new TSK 0
I20260709 12:16:34.168143 25028 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:34.197299 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:34.204052 25038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:34.205124 25039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:34.209339 24421 server_base.cc:1061] running on GCE node
W20260709 12:16:34.209282 25041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:34.211032 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:34.211304 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:34.211480 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599394211461 us; error 0 us; skew 500 ppm
I20260709 12:16:34.212031 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:34.214677 24421 webserver.cc:533] Webserver started at http://127.23.217.65:37863/ using document root <none> and password file <none>
I20260709 12:16:34.215167 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:34.215381 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:34.215664 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:34.216732 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "6483f8987428479a99e8e1c6e6be4d49"
format_stamp: "Formatted at 2026-07-09 12:16:34 on dist-test-slave-lg8f"
I20260709 12:16:34.221226 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:34.224613 25046 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.225358 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:34.225661 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "6483f8987428479a99e8e1c6e6be4d49"
format_stamp: "Formatted at 2026-07-09 12:16:34 on dist-test-slave-lg8f"
I20260709 12:16:34.225939 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:34.244607 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:34.245812 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:34.247339 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:34.249648 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:34.249873 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.250113 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:34.250293 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.295094 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:33669
I20260709 12:16:34.295249 25108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:33669 every 8 connection(s)
I20260709 12:16:34.300498 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:34.308818 25113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:34.310282 25114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:34.313592 25116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:34.314788 24421 server_base.cc:1061] running on GCE node
I20260709 12:16:34.316260 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:34.316485 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:34.316670 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599394316650 us; error 0 us; skew 500 ppm
I20260709 12:16:34.317193 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:34.319718 24421 webserver.cc:533] Webserver started at http://127.23.217.66:38689/ using document root <none> and password file <none>
I20260709 12:16:34.320204 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:34.320408 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:34.320691 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:34.321799 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "5888170dcea146da989a6993c1612df1"
format_stamp: "Formatted at 2026-07-09 12:16:34 on dist-test-slave-lg8f"
I20260709 12:16:34.327034 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:16:34.327721 25109 heartbeater.cc:344] Connected to a master server at 127.23.217.126:39547
I20260709 12:16:34.328161 25109 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:34.328914 25109 heartbeater.cc:507] Master 127.23.217.126:39547 requested a full tablet report, sending...
I20260709 12:16:34.331173 25121 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.331353 24982 ts_manager.cc:194] Registered new tserver with Master: 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669)
I20260709 12:16:34.332175 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:16:34.332544 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "5888170dcea146da989a6993c1612df1"
format_stamp: "Formatted at 2026-07-09 12:16:34 on dist-test-slave-lg8f"
I20260709 12:16:34.332927 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:34.334082 24982 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50398
I20260709 12:16:34.364493 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:34.365947 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:34.367632 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:34.370137 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:34.370332 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.370677 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:34.370831 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.419711 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:37371
I20260709 12:16:34.419811 25183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:37371 every 8 connection(s)
I20260709 12:16:34.425110 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:34.435868 25187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:34.440825 25184 heartbeater.cc:344] Connected to a master server at 127.23.217.126:39547
W20260709 12:16:34.441140 25188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:34.441433 25184 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:34.442622 25184 heartbeater.cc:507] Master 127.23.217.126:39547 requested a full tablet report, sending...
W20260709 12:16:34.442934 25190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:34.442622 24421 server_base.cc:1061] running on GCE node
I20260709 12:16:34.444758 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:34.445027 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:34.445232 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599394445211 us; error 0 us; skew 500 ppm
I20260709 12:16:34.445381 24982 ts_manager.cc:194] Registered new tserver with Master: 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:34.445881 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:34.447090 24982 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50402
I20260709 12:16:34.449134 24421 webserver.cc:533] Webserver started at http://127.23.217.67:39697/ using document root <none> and password file <none>
I20260709 12:16:34.449735 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:34.449961 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:34.450244 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:34.451570 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "3feccf1b43484fcfb09ad15e894e901f"
format_stamp: "Formatted at 2026-07-09 12:16:34 on dist-test-slave-lg8f"
I20260709 12:16:34.456364 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:16:34.460041 25195 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.461054 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:16:34.461418 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "3feccf1b43484fcfb09ad15e894e901f"
format_stamp: "Formatted at 2026-07-09 12:16:34 on dist-test-slave-lg8f"
I20260709 12:16:34.461758 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:34.491194 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:34.492758 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:34.494621 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:34.497218 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:34.497486 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.497776 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:34.497979 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:34.544902 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:42691
I20260709 12:16:34.545011 25257 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:42691 every 8 connection(s)
I20260709 12:16:34.559844 25258 heartbeater.cc:344] Connected to a master server at 127.23.217.126:39547
I20260709 12:16:34.560278 25258 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:34.561122 25258 heartbeater.cc:507] Master 127.23.217.126:39547 requested a full tablet report, sending...
I20260709 12:16:34.563759 24982 ts_manager.cc:194] Registered new tserver with Master: 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:34.564249 24421 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016012449s
I20260709 12:16:34.565423 24982 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50416
I20260709 12:16:34.593549 24982 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50426:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:34.596967 24982 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:34.688436 25149 tablet_service.cc:1511] Processing CreateTablet for tablet e893ccf562714101b57af43c7f136787 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:34.689299 25147 tablet_service.cc:1511] Processing CreateTablet for tablet a2a68fa2ab654a65b609f2e04a7111c3 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:34.689615 25146 tablet_service.cc:1511] Processing CreateTablet for tablet 6075ea7b15354e628f157bd3c2fffb24 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:34.690189 25149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e893ccf562714101b57af43c7f136787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.690866 25147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2a68fa2ab654a65b609f2e04a7111c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.692104 25146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6075ea7b15354e628f157bd3c2fffb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.692360 25148 tablet_service.cc:1511] Processing CreateTablet for tablet 1b5256fb4c554d58944c09a672b974a2 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:34.693684 25148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5256fb4c554d58944c09a672b974a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.697503 25074 tablet_service.cc:1511] Processing CreateTablet for tablet e893ccf562714101b57af43c7f136787 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:34.698918 25074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e893ccf562714101b57af43c7f136787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.703742 25073 tablet_service.cc:1511] Processing CreateTablet for tablet 1b5256fb4c554d58944c09a672b974a2 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:34.705137 25073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5256fb4c554d58944c09a672b974a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.708000 25072 tablet_service.cc:1511] Processing CreateTablet for tablet a2a68fa2ab654a65b609f2e04a7111c3 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:34.708788 25071 tablet_service.cc:1511] Processing CreateTablet for tablet 6075ea7b15354e628f157bd3c2fffb24 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:34.709381 25072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2a68fa2ab654a65b609f2e04a7111c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.710321 25071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6075ea7b15354e628f157bd3c2fffb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.727787 25223 tablet_service.cc:1511] Processing CreateTablet for tablet e893ccf562714101b57af43c7f136787 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:34.729393 25223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e893ccf562714101b57af43c7f136787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.731415 25070 tablet_service.cc:1511] Processing CreateTablet for tablet 2e8490ab48874dccaed8b9cfc46427df (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:34.731778 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 1b5256fb4c554d58944c09a672b974a2 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:34.732882 25070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e8490ab48874dccaed8b9cfc46427df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.733253 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5256fb4c554d58944c09a672b974a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.735720 25221 tablet_service.cc:1511] Processing CreateTablet for tablet a2a68fa2ab654a65b609f2e04a7111c3 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:34.737182 25221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2a68fa2ab654a65b609f2e04a7111c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.739749 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 6075ea7b15354e628f157bd3c2fffb24 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:34.743395 25219 tablet_service.cc:1511] Processing CreateTablet for tablet 2e8490ab48874dccaed8b9cfc46427df (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:34.747367 25273 tablet_bootstrap.cc:492] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:34.750902 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6075ea7b15354e628f157bd3c2fffb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.752673 25219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e8490ab48874dccaed8b9cfc46427df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.760906 25146 tablet_service.cc:1511] Processing CreateTablet for tablet 81b3ee815a2a40a89e20379f5b91daa0 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:34.762256 25146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b3ee815a2a40a89e20379f5b91daa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.762748 25218 tablet_service.cc:1511] Processing CreateTablet for tablet 81b3ee815a2a40a89e20379f5b91daa0 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:34.764292 25218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b3ee815a2a40a89e20379f5b91daa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.764720 25071 tablet_service.cc:1511] Processing CreateTablet for tablet a0437decc4cc4ee1b6d0ee3fbb48a32f (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:34.766203 25071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0437decc4cc4ee1b6d0ee3fbb48a32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.767076 25148 tablet_service.cc:1511] Processing CreateTablet for tablet a0437decc4cc4ee1b6d0ee3fbb48a32f (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:34.768349 25148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0437decc4cc4ee1b6d0ee3fbb48a32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.768545 25217 tablet_service.cc:1511] Processing CreateTablet for tablet a0437decc4cc4ee1b6d0ee3fbb48a32f (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:34.769626 25273 tablet_bootstrap.cc:654] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.769907 25217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0437decc4cc4ee1b6d0ee3fbb48a32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.758720 25070 tablet_service.cc:1511] Processing CreateTablet for tablet 81b3ee815a2a40a89e20379f5b91daa0 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:34.783551 25070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b3ee815a2a40a89e20379f5b91daa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.779726 25147 tablet_service.cc:1511] Processing CreateTablet for tablet 2e8490ab48874dccaed8b9cfc46427df (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:34.794322 25147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e8490ab48874dccaed8b9cfc46427df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.797835 25073 tablet_service.cc:1511] Processing CreateTablet for tablet 377d4faa6ec445fcb31987aac4aeebe4 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:34.799278 25073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377d4faa6ec445fcb31987aac4aeebe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.799715 25149 tablet_service.cc:1511] Processing CreateTablet for tablet 377d4faa6ec445fcb31987aac4aeebe4 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:34.801017 25149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377d4faa6ec445fcb31987aac4aeebe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.816246 25216 tablet_service.cc:1511] Processing CreateTablet for tablet 377d4faa6ec445fcb31987aac4aeebe4 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:34.817663 25216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377d4faa6ec445fcb31987aac4aeebe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.832036 25276 tablet_bootstrap.cc:492] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:34.832630 25274 tablet_bootstrap.cc:492] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:34.843959 25274 tablet_bootstrap.cc:654] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.847607 25071 tablet_service.cc:1511] Processing CreateTablet for tablet d997f329a972472fb5bec13119cb2c52 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:34.849056 25071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d997f329a972472fb5bec13119cb2c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.849817 25148 tablet_service.cc:1511] Processing CreateTablet for tablet d997f329a972472fb5bec13119cb2c52 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:34.851158 25148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d997f329a972472fb5bec13119cb2c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.851977 25222 tablet_service.cc:1511] Processing CreateTablet for tablet d997f329a972472fb5bec13119cb2c52 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:34.853363 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d997f329a972472fb5bec13119cb2c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.855299 25145 tablet_service.cc:1511] Processing CreateTablet for tablet 5495c5a44e6e4af0859dfb93da9c4605 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:34.856676 25145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5495c5a44e6e4af0859dfb93da9c4605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.856957 25074 tablet_service.cc:1511] Processing CreateTablet for tablet 5495c5a44e6e4af0859dfb93da9c4605 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:34.858235 25074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5495c5a44e6e4af0859dfb93da9c4605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.866607 25273 tablet_bootstrap.cc:492] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:34.867187 25273 ts_tablet_manager.cc:1403] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.036s
I20260709 12:16:34.870337 25273 raft_consensus.cc:359] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.871140 25273 raft_consensus.cc:385] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.871491 25273 raft_consensus.cc:740] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.879263 25276 tablet_bootstrap.cc:654] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.886991 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 5495c5a44e6e4af0859dfb93da9c4605 (DEFAULT_TABLE table=test-workload [id=87f424921c564238ac5d900203471b40]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:34.888391 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5495c5a44e6e4af0859dfb93da9c4605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:34.892885 25273 consensus_queue.cc:260] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.910157 25184 heartbeater.cc:499] Master 127.23.217.126:39547 was elected leader, sending a full tablet report...
I20260709 12:16:34.910521 25273 ts_tablet_manager.cc:1434] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.011s
I20260709 12:16:34.911685 25273 tablet_bootstrap.cc:492] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:34.913172 25274 tablet_bootstrap.cc:492] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:34.913640 25274 ts_tablet_manager.cc:1403] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.013s
I20260709 12:16:34.916535 25274 raft_consensus.cc:359] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:34.917276 25274 raft_consensus.cc:385] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.917834 25274 raft_consensus.cc:740] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.920071 25274 consensus_queue.cc:260] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:34.926465 25276 tablet_bootstrap.cc:492] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:34.927033 25276 ts_tablet_manager.cc:1403] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.115s	user 0.007s	sys 0.017s
I20260709 12:16:34.930006 25273 tablet_bootstrap.cc:654] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.930557 25274 ts_tablet_manager.cc:1434] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 12:16:34.930373 25276 raft_consensus.cc:359] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.931310 25276 raft_consensus.cc:385] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.931383 25109 heartbeater.cc:499] Master 127.23.217.126:39547 was elected leader, sending a full tablet report...
I20260709 12:16:34.931713 25276 raft_consensus.cc:740] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.932731 25276 consensus_queue.cc:260] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.936627 25274 tablet_bootstrap.cc:492] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:34.943683 25274 tablet_bootstrap.cc:654] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.948314 25258 heartbeater.cc:499] Master 127.23.217.126:39547 was elected leader, sending a full tablet report...
I20260709 12:16:34.949013 25276 ts_tablet_manager.cc:1434] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.015s
I20260709 12:16:34.949191 25273 tablet_bootstrap.cc:492] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:34.949682 25273 ts_tablet_manager.cc:1403] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.011s
I20260709 12:16:34.950103 25276 tablet_bootstrap.cc:492] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:34.952765 25273 raft_consensus.cc:359] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.954341 25273 raft_consensus.cc:385] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.954704 25273 raft_consensus.cc:740] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.955628 25273 consensus_queue.cc:260] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.957296 25276 tablet_bootstrap.cc:654] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.958238 25273 ts_tablet_manager.cc:1434] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:16:34.959755 25273 tablet_bootstrap.cc:492] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:34.966125 25274 tablet_bootstrap.cc:492] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:34.966686 25274 ts_tablet_manager.cc:1403] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.002s
I20260709 12:16:34.968997 25273 tablet_bootstrap.cc:654] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.968940 25274 raft_consensus.cc:359] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.969601 25274 raft_consensus.cc:385] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.969875 25274 raft_consensus.cc:740] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.970556 25274 consensus_queue.cc:260] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.975251 25274 ts_tablet_manager.cc:1434] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 12:16:34.976110 25274 tablet_bootstrap.cc:492] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:34.977687 25276 tablet_bootstrap.cc:492] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:34.978356 25276 ts_tablet_manager.cc:1403] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.008s
I20260709 12:16:34.982445 25276 raft_consensus.cc:359] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.983276 25276 raft_consensus.cc:385] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.982440 25274 tablet_bootstrap.cc:654] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.983536 25276 raft_consensus.cc:740] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.984717 25276 consensus_queue.cc:260] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.985320 25273 tablet_bootstrap.cc:492] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:34.985924 25273 ts_tablet_manager.cc:1403] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.011s
I20260709 12:16:34.987310 25276 ts_tablet_manager.cc:1434] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:16:34.988268 25276 tablet_bootstrap.cc:492] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:34.989111 25273 raft_consensus.cc:359] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.989889 25273 raft_consensus.cc:385] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.990238 25273 raft_consensus.cc:740] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.991235 25273 consensus_queue.cc:260] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:34.995784 25276 tablet_bootstrap.cc:654] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.996742 25273 ts_tablet_manager.cc:1434] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 12:16:34.997943 25273 tablet_bootstrap.cc:492] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:35.004298 25274 tablet_bootstrap.cc:492] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.004940 25274 ts_tablet_manager.cc:1403] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 12:16:35.005759 25273 tablet_bootstrap.cc:654] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.006235 25276 tablet_bootstrap.cc:492] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.006811 25276 ts_tablet_manager.cc:1403] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:16:35.008052 25274 raft_consensus.cc:359] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.008837 25274 raft_consensus.cc:385] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.009145 25274 raft_consensus.cc:740] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.009907 25274 consensus_queue.cc:260] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.010170 25276 raft_consensus.cc:359] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.011193 25276 raft_consensus.cc:385] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.011592 25276 raft_consensus.cc:740] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.012141 25274 ts_tablet_manager.cc:1434] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:35.013259 25274 tablet_bootstrap.cc:492] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:35.012872 25276 consensus_queue.cc:260] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.019737 25276 ts_tablet_manager.cc:1434] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 12:16:35.019919 25274 tablet_bootstrap.cc:654] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.021036 25276 tablet_bootstrap.cc:492] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:35.022181 25282 raft_consensus.cc:493] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.022727 25282 raft_consensus.cc:515] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.028945 25282 leader_election.cc:290] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.029682 25276 tablet_bootstrap.cc:654] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.047564 25084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8490ab48874dccaed8b9cfc46427df" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.048961 25276 tablet_bootstrap.cc:492] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.049533 25276 ts_tablet_manager.cc:1403] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.007s
W20260709 12:16:35.049925 25199 leader_election.cc:343] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.052673 25276 raft_consensus.cc:359] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.053403 25276 raft_consensus.cc:385] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.053694 25276 raft_consensus.cc:740] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.054690 25276 consensus_queue.cc:260] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.057101 25278 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.057569 25278 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.057983 25282 raft_consensus.cc:493] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.058434 25282 raft_consensus.cc:515] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.060385 25282 leader_election.cc:290] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.062376 25276 ts_tablet_manager.cc:1434] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 12:16:35.063498 25276 tablet_bootstrap.cc:492] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:35.064433 25278 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.075083 25084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.076345 25273 tablet_bootstrap.cc:492] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:35.076500 25280 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.076874 25273 ts_tablet_manager.cc:1403] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.010s
I20260709 12:16:35.077036 25280 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
W20260709 12:16:35.076782 25199 leader_election.cc:343] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.077970 25274 tablet_bootstrap.cc:492] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.078455 25274 ts_tablet_manager.cc:1403] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.065s	user 0.026s	sys 0.021s
I20260709 12:16:35.081068 25280 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.082613 25274 raft_consensus.cc:359] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.085391 25274 raft_consensus.cc:385] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.087255 25274 raft_consensus.cc:740] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.087410 25159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8490ab48874dccaed8b9cfc46427df" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.087687 25276 tablet_bootstrap.cc:654] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.091601 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.091856 25273 raft_consensus.cc:359] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.092578 25273 raft_consensus.cc:385] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.092896 25273 raft_consensus.cc:740] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.093698 25273 consensus_queue.cc:260] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.096489 25273 ts_tablet_manager.cc:1434] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:16:35.097678 25273 tablet_bootstrap.cc:492] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:35.103399 25278 raft_consensus.cc:493] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.104120 25278 raft_consensus.cc:515] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.104946 25273 tablet_bootstrap.cc:654] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.111078 25274 consensus_queue.cc:260] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
W20260709 12:16:35.111738 25198 leader_election.cc:343] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.112416 25198 leader_election.cc:304] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f; no voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.113391 25282 raft_consensus.cc:2749] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:35.113801 25198 leader_election.cc:343] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.113456 25274 ts_tablet_manager.cc:1434] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:16:35.107671 25278 leader_election.cc:290] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.117436 25278 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.114212 25198 leader_election.cc:304] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f; no voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.119071 25282 raft_consensus.cc:2749] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.121125 25274 tablet_bootstrap.cc:492] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:35.118059 25278 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
W20260709 12:16:35.123957 25030 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:35.134801 25278 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.141450 25084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.142781 25282 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.144431 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.144538 25282 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.145301 25082 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.145291 25083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.146683 25083 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.149320 25125 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.150581 25125 leader_election.cc:304] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.150718 25278 raft_consensus.cc:2804] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.151176 25278 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.151595 25278 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.151962 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.150553 25276 tablet_bootstrap.cc:492] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.152145 25280 raft_consensus.cc:493] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.152798 25276 ts_tablet_manager.cc:1403] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.090s	user 0.029s	sys 0.029s
I20260709 12:16:35.152889 25280 raft_consensus.cc:515] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.147817 25084 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.155055 25280 leader_election.cc:290] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.155741 25276 raft_consensus.cc:359] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.156404 25276 raft_consensus.cc:385] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.156680 25276 raft_consensus.cc:740] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.157397 25276 consensus_queue.cc:260] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.148564 25282 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.158964 25159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.159543 25125 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.159788 25159 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 0.
I20260709 12:16:35.160966 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e893ccf562714101b57af43c7f136787" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.161759 25157 raft_consensus.cc:2468] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 0.
I20260709 12:16:35.161751 25049 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.167934 25280 raft_consensus.cc:2804] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.168294 25280 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.168633 25280 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.168548 25049 leader_election.cc:304] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.173347 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.176613 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.177322 25233 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.172029 25084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.165597 25299 raft_consensus.cc:2804] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.180656 25299 raft_consensus.cc:493] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.178566 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
W20260709 12:16:35.181156 25122 leader_election.cc:343] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.180123 25084 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 0.
I20260709 12:16:35.183285 25273 tablet_bootstrap.cc:492] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:35.184484 25274 tablet_bootstrap.cc:654] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.187335 25232 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.187805 25273 ts_tablet_manager.cc:1403] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.090s	user 0.019s	sys 0.029s
I20260709 12:16:35.187284 25301 raft_consensus.cc:2804] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.188958 25301 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.186625 25299 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.189397 25301 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.184608 25280 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.187085 25302 raft_consensus.cc:2804] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.190661 25302 raft_consensus.cc:493] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.191051 25302 raft_consensus.cc:3060] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.191416 25199 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.192306 25282 raft_consensus.cc:2804] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.192682 25282 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.193025 25282 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.192868 25273 raft_consensus.cc:359] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.193616 25273 raft_consensus.cc:385] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.193946 25273 raft_consensus.cc:740] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.196970 25276 ts_tablet_manager.cc:1434] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.044s	user 0.011s	sys 0.010s
I20260709 12:16:35.205449 25276 tablet_bootstrap.cc:492] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:35.206056 25273 consensus_queue.cc:260] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.208244 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.209692 25299 raft_consensus.cc:515] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.210798 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e893ccf562714101b57af43c7f136787" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.212321 25280 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.212704 25276 tablet_bootstrap.cc:654] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.213348 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1"
W20260709 12:16:35.215508 25047 leader_election.cc:343] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:35.216706 25047 leader_election.cc:343] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.218499 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.219075 25282 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
W20260709 12:16:35.220113 25047 leader_election.cc:343] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.225409 25299 leader_election.cc:290] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.223126 25278 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.226980 25084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:35.227185 25083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:35.235407 25084 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.235551 25083 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.235913 25159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:35.238852 25300 raft_consensus.cc:493] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.239437 25300 raft_consensus.cc:515] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.242838 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.243264 25084 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 1.
I20260709 12:16:35.227726 25304 raft_consensus.cc:493] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.226980 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.231144 25274 tablet_bootstrap.cc:492] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.246866 25199 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 5888170dcea146da989a6993c1612df1
I20260709 12:16:35.247092 25274 ts_tablet_manager.cc:1403] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.024s
I20260709 12:16:35.246937 25232 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.249221 25159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.250425 25274 raft_consensus.cc:359] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
W20260709 12:16:35.250799 25198 leader_election.cc:343] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.251202 25274 raft_consensus.cc:385] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.251771 25274 raft_consensus.cc:740] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.251078 25304 raft_consensus.cc:515] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.252784 25274 consensus_queue.cc:260] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.254194 25300 leader_election.cc:290] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.255262 25300 raft_consensus.cc:2804] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.256115 25274 ts_tablet_manager.cc:1434] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:35.256294 25300 raft_consensus.cc:697] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 LEADER]: Becoming Leader. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:35.257661 25274 tablet_bootstrap.cc:492] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:35.257809 25300 consensus_queue.cc:237] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.233108 25273 ts_tablet_manager.cc:1434] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 12:16:35.265527 25274 tablet_bootstrap.cc:654] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.265429 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.265923 25273 tablet_bootstrap.cc:492] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
W20260709 12:16:35.267140 25122 leader_election.cc:343] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.265429 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.233940 25282 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.276871 25158 raft_consensus.cc:2393] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3feccf1b43484fcfb09ad15e894e901f in current term 1: Already voted for candidate 5888170dcea146da989a6993c1612df1 in this term.
W20260709 12:16:35.277261 25199 leader_election.cc:343] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.277763 25199 leader_election.cc:304] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f; no voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.271416 25304 leader_election.cc:290] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
W20260709 12:16:35.275127 25125 leader_election.cc:343] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.284658 25083 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:35.286012 25276 tablet_bootstrap.cc:492] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.286475 25276 ts_tablet_manager.cc:1403] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.009s
I20260709 12:16:35.288614 25125 leader_election.cc:304] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5888170dcea146da989a6993c1612df1; no voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.289217 25276 raft_consensus.cc:359] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.290012 25276 raft_consensus.cc:385] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.291131 25125 leader_election.cc:304] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.292212 25299 raft_consensus.cc:2749] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.292333 25304 raft_consensus.cc:2804] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.293083 25304 raft_consensus.cc:697] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Becoming Leader. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:35.294253 25273 tablet_bootstrap.cc:654] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.295523 25301 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.290310 25276 raft_consensus.cc:740] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.297562 25157 raft_consensus.cc:2393] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6483f8987428479a99e8e1c6e6be4d49 in current term 1: Already voted for candidate 5888170dcea146da989a6993c1612df1 in this term.
I20260709 12:16:35.279968 25282 raft_consensus.cc:2749] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.298316 25276 consensus_queue.cc:260] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.271639 25302 raft_consensus.cc:515] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.302973 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.301080 25049 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6483f8987428479a99e8e1c6e6be4d49; no voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1
I20260709 12:16:35.301682 25083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:35.297554 25232 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:35.299717 25278 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.304965 25280 raft_consensus.cc:2749] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.306667 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e893ccf562714101b57af43c7f136787" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.306981 25083 raft_consensus.cc:2393] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5888170dcea146da989a6993c1612df1 in current term 1: Already voted for candidate 3feccf1b43484fcfb09ad15e894e901f in this term.
I20260709 12:16:35.304993 25302 leader_election.cc:290] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.304924 25276 ts_tablet_manager.cc:1434] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 12:16:35.308272 25125 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5888170dcea146da989a6993c1612df1; no voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.309417 25276 tablet_bootstrap.cc:492] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:35.310288 25278 raft_consensus.cc:2749] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.310701 25274 tablet_bootstrap.cc:492] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.311197 25274 ts_tablet_manager.cc:1403] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.025s
I20260709 12:16:35.310506 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:35.313727 25082 raft_consensus.cc:2393] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5888170dcea146da989a6993c1612df1 in current term 1: Already voted for candidate 6483f8987428479a99e8e1c6e6be4d49 in this term.
I20260709 12:16:35.308411 25301 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.314385 25274 raft_consensus.cc:359] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.308414 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e893ccf562714101b57af43c7f136787" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:35.315174 25274 raft_consensus.cc:385] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.315371 25157 raft_consensus.cc:3060] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.315495 25274 raft_consensus.cc:740] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.316298 25274 consensus_queue.cc:260] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.309471 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.318558 25274 ts_tablet_manager.cc:1434] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
W20260709 12:16:35.319743 25122 leader_election.cc:343] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.320616 25274 tablet_bootstrap.cc:492] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:35.320426 25122 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5888170dcea146da989a6993c1612df1; no voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49
W20260709 12:16:35.322510 25047 leader_election.cc:343] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.323745 25276 tablet_bootstrap.cc:654] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.322225 25301 raft_consensus.cc:2749] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.325449 25304 consensus_queue.cc:237] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.331772 24982 catalog_manager.cc:5697] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3feccf1b43484fcfb09ad15e894e901f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:35.335852 25273 tablet_bootstrap.cc:492] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:35.336862 25273 ts_tablet_manager.cc:1403] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.071s	user 0.016s	sys 0.000s
I20260709 12:16:35.341185 25274 tablet_bootstrap.cc:654] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.341960 25157 raft_consensus.cc:2468] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 1.
I20260709 12:16:35.345804 25273 raft_consensus.cc:359] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.346915 25273 raft_consensus.cc:385] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.347266 25273 raft_consensus.cc:740] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.348006 25273 consensus_queue.cc:260] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.349923 25276 tablet_bootstrap.cc:492] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.350193 25273 ts_tablet_manager.cc:1434] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:16:35.350558 25276 ts_tablet_manager.cc:1403] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.019s
I20260709 12:16:35.351198 25273 tablet_bootstrap.cc:492] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:35.353510 25276 raft_consensus.cc:359] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.354254 25276 raft_consensus.cc:385] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.355612 25049 leader_election.cc:304] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 3feccf1b43484fcfb09ad15e894e901f
I20260709 12:16:35.357004 25302 raft_consensus.cc:2804] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.357614 25302 raft_consensus.cc:697] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:35.358742 25302 consensus_queue.cc:237] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.354650 25276 raft_consensus.cc:740] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.364858 25276 consensus_queue.cc:260] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.368650 25273 tablet_bootstrap.cc:654] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.370682 25276 ts_tablet_manager.cc:1434] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:16:35.372053 25276 tablet_bootstrap.cc:492] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:35.373047 24978 catalog_manager.cc:5697] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5888170dcea146da989a6993c1612df1 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "5888170dcea146da989a6993c1612df1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.380162 25274 tablet_bootstrap.cc:492] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.381174 25274 ts_tablet_manager.cc:1403] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.013s
I20260709 12:16:35.380419 24979 catalog_manager.cc:5697] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:35.384688 25273 tablet_bootstrap.cc:492] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:35.384385 25274 raft_consensus.cc:359] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.385078 25276 tablet_bootstrap.cc:654] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.385103 25274 raft_consensus.cc:385] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.385181 25273 ts_tablet_manager.cc:1403] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.002s
I20260709 12:16:35.385869 25274 raft_consensus.cc:740] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.386837 25274 consensus_queue.cc:260] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.389827 25304 raft_consensus.cc:493] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.389467 25273 raft_consensus.cc:359] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.390430 25304 raft_consensus.cc:515] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.391100 25273 raft_consensus.cc:385] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.391824 25273 raft_consensus.cc:740] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.393235 25273 consensus_queue.cc:260] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.394292 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
W20260709 12:16:35.395898 25125 leader_election.cc:343] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.397291 25304 leader_election.cc:290] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.397714 25273 ts_tablet_manager.cc:1434] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.007s
I20260709 12:16:35.397964 25274 ts_tablet_manager.cc:1434] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:16:35.397894 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.398691 25233 raft_consensus.cc:2468] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.399111 25274 tablet_bootstrap.cc:492] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:35.399659 25273 tablet_bootstrap.cc:492] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:35.399919 25122 leader_election.cc:304] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.401081 25304 raft_consensus.cc:2804] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.401466 25304 raft_consensus.cc:493] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.401805 25304 raft_consensus.cc:3060] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.405869 25274 tablet_bootstrap.cc:654] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.411882 25273 tablet_bootstrap.cc:654] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.413702 25304 raft_consensus.cc:515] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.414283 25276 tablet_bootstrap.cc:492] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.415047 25276 ts_tablet_manager.cc:1403] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.017s
I20260709 12:16:35.417145 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
W20260709 12:16:35.418857 25125 leader_election.cc:343] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.420522 25276 raft_consensus.cc:359] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.421445 25276 raft_consensus.cc:385] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.421837 25276 raft_consensus.cc:740] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.423175 25274 tablet_bootstrap.cc:492] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.423688 25274 ts_tablet_manager.cc:1403] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.004s
I20260709 12:16:35.424507 25276 consensus_queue.cc:260] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.425796 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.426385 25233 raft_consensus.cc:3060] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.426214 25274 raft_consensus.cc:359] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.426905 25274 raft_consensus.cc:385] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.427183 25274 raft_consensus.cc:740] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.428084 25274 consensus_queue.cc:260] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.428916 25276 ts_tablet_manager.cc:1434] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:16:35.430389 25274 ts_tablet_manager.cc:1434] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:35.431803 25274 tablet_bootstrap.cc:492] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:35.431890 25276 tablet_bootstrap.cc:492] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:35.432546 25273 tablet_bootstrap.cc:492] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:35.433048 25273 ts_tablet_manager.cc:1403] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.010s
I20260709 12:16:35.436301 25233 raft_consensus.cc:2468] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:35.436314 25273 raft_consensus.cc:359] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.437312 25273 raft_consensus.cc:385] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.437649 25273 raft_consensus.cc:740] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.437717 25122 leader_election.cc:304] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.437649 25274 tablet_bootstrap.cc:654] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.438741 25301 raft_consensus.cc:2804] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.444823 25273 consensus_queue.cc:260] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.446339 25304 leader_election.cc:290] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.450636 25276 tablet_bootstrap.cc:654] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.450714 25301 raft_consensus.cc:697] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Becoming Leader. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:35.452350 25301 consensus_queue.cc:237] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.456130 25274 tablet_bootstrap.cc:492] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.456629 25274 ts_tablet_manager.cc:1403] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.005s
I20260709 12:16:35.459386 25274 raft_consensus.cc:359] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.460122 25274 raft_consensus.cc:385] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.460379 25274 raft_consensus.cc:740] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.461213 25273 ts_tablet_manager.cc:1434] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:16:35.461179 25274 consensus_queue.cc:260] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.462595 25273 tablet_bootstrap.cc:492] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:35.465636 25274 ts_tablet_manager.cc:1434] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:16:35.466614 25274 tablet_bootstrap.cc:492] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49: Bootstrap starting.
I20260709 12:16:35.472621 25274 tablet_bootstrap.cc:654] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.474849 25302 raft_consensus.cc:493] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.475322 25302 raft_consensus.cc:515] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.476102 25276 tablet_bootstrap.cc:492] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.476680 25276 ts_tablet_manager.cc:1403] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.006s
I20260709 12:16:35.477370 25273 tablet_bootstrap.cc:654] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.477500 25302 leader_election.cc:290] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.479306 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5495c5a44e6e4af0859dfb93da9c4605" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.479712 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5495c5a44e6e4af0859dfb93da9c4605" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.475339 24982 catalog_manager.cc:5697] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5888170dcea146da989a6993c1612df1 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "5888170dcea146da989a6993c1612df1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:35.481163 25049 leader_election.cc:343] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.481178 25300 raft_consensus.cc:493] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:35.482244 25047 leader_election.cc:343] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.482174 25300 raft_consensus.cc:515] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.482905 25047 leader_election.cc:304] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6483f8987428479a99e8e1c6e6be4d49; no voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1
I20260709 12:16:35.484362 25302 raft_consensus.cc:2749] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.487296 25274 tablet_bootstrap.cc:492] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49: No bootstrap required, opened a new log
I20260709 12:16:35.487388 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997f329a972472fb5bec13119cb2c52" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.487887 25274 ts_tablet_manager.cc:1403] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.000s
W20260709 12:16:35.489074 25198 leader_election.cc:343] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.489338 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997f329a972472fb5bec13119cb2c52" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.489776 25300 leader_election.cc:290] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.482152 25276 raft_consensus.cc:359] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
W20260709 12:16:35.490891 25199 leader_election.cc:343] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.489343 25273 tablet_bootstrap.cc:492] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:35.491020 25274 raft_consensus.cc:359] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.491421 25199 leader_election.cc:304] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f; no voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.491925 25274 raft_consensus.cc:385] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.492003 25273 ts_tablet_manager.cc:1403] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.008s
I20260709 12:16:35.492333 25274 raft_consensus.cc:740] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.492589 25300 raft_consensus.cc:2749] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.491164 25276 raft_consensus.cc:385] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.493284 25276 raft_consensus.cc:740] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.493099 25274 consensus_queue.cc:260] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.494136 25276 consensus_queue.cc:260] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.495500 25274 ts_tablet_manager.cc:1434] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:16:35.496522 25276 ts_tablet_manager.cc:1434] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:35.501120 25276 tablet_bootstrap.cc:492] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f: Bootstrap starting.
I20260709 12:16:35.502170 25273 raft_consensus.cc:359] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.503067 25273 raft_consensus.cc:385] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.503468 25273 raft_consensus.cc:740] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.504109 25273 consensus_queue.cc:260] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.505786 25273 ts_tablet_manager.cc:1434] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.000s
I20260709 12:16:35.507256 25276 tablet_bootstrap.cc:654] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.507552 25273 tablet_bootstrap.cc:492] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1: Bootstrap starting.
I20260709 12:16:35.514986 25273 tablet_bootstrap.cc:654] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.519668 25276 tablet_bootstrap.cc:492] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f: No bootstrap required, opened a new log
I20260709 12:16:35.520228 25276 ts_tablet_manager.cc:1403] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.001s
I20260709 12:16:35.520429 25273 tablet_bootstrap.cc:492] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1: No bootstrap required, opened a new log
I20260709 12:16:35.520900 25273 ts_tablet_manager.cc:1403] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 12:16:35.523399 25276 raft_consensus.cc:359] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.523643 25273 raft_consensus.cc:359] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.524123 25276 raft_consensus.cc:385] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.524186 25273 raft_consensus.cc:385] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.524493 25276 raft_consensus.cc:740] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.524524 25273 raft_consensus.cc:740] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.525269 25273 consensus_queue.cc:260] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.525269 25276 consensus_queue.cc:260] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.527486 25273 ts_tablet_manager.cc:1434] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:16:35.529477 25276 ts_tablet_manager.cc:1434] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:35.544176 25302 raft_consensus.cc:493] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.544682 25302 raft_consensus.cc:515] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.546345 25302 leader_election.cc:290] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.547534 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.547504 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.548309 25233 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 0.
I20260709 12:16:35.548565 25157 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 0.
I20260709 12:16:35.549297 25047 leader_election.cc:304] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.549974 25302 raft_consensus.cc:2804] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.550341 25302 raft_consensus.cc:493] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.550717 25302 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.556275 25302 raft_consensus.cc:515] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.558466 25302 leader_election.cc:290] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.559582 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.559600 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:35.560089 25233 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.560397 25157 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.565465 25233 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 1.
I20260709 12:16:35.566046 25157 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 1.
I20260709 12:16:35.566710 25047 leader_election.cc:304] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.567584 25302 raft_consensus.cc:2804] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.567963 25302 raft_consensus.cc:697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:35.568737 25302 consensus_queue.cc:237] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.576560 24978 catalog_manager.cc:5697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.616950 25302 raft_consensus.cc:493] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.617532 25302 raft_consensus.cc:515] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.619745 25302 leader_election.cc:290] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.620735 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8490ab48874dccaed8b9cfc46427df" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.620910 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8490ab48874dccaed8b9cfc46427df" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.621304 25157 raft_consensus.cc:2468] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 0.
I20260709 12:16:35.621608 25233 raft_consensus.cc:2468] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 0.
I20260709 12:16:35.622359 25049 leader_election.cc:304] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.623184 25302 raft_consensus.cc:2804] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.623555 25302 raft_consensus.cc:493] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.623852 25302 raft_consensus.cc:3060] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.629009 25302 raft_consensus.cc:515] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.630775 25302 leader_election.cc:290] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.631479 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8490ab48874dccaed8b9cfc46427df" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:35.631701 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e8490ab48874dccaed8b9cfc46427df" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.631965 25157 raft_consensus.cc:3060] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.632288 25233 raft_consensus.cc:3060] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.637115 25233 raft_consensus.cc:2468] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 1.
I20260709 12:16:35.637538 25157 raft_consensus.cc:2468] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 1.
I20260709 12:16:35.638015 25047 leader_election.cc:304] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.638775 25302 raft_consensus.cc:2804] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.639273 25302 raft_consensus.cc:697] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:35.639578 25280 raft_consensus.cc:493] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.640062 25280 raft_consensus.cc:515] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.640323 25302 consensus_queue.cc:237] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.641059 25300 raft_consensus.cc:493] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.641551 25300 raft_consensus.cc:515] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } } peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } }
I20260709 12:16:35.642292 25280 leader_election.cc:290] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.643299 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.643596 25300 leader_election.cc:290] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669)
I20260709 12:16:35.643818 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.644564 25233 raft_consensus.cc:2393] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6483f8987428479a99e8e1c6e6be4d49 in current term 1: Already voted for candidate 5888170dcea146da989a6993c1612df1 in this term.
I20260709 12:16:35.644385 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e893ccf562714101b57af43c7f136787" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.645612 25158 raft_consensus.cc:2393] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3feccf1b43484fcfb09ad15e894e901f in current term 1: Already voted for candidate 6483f8987428479a99e8e1c6e6be4d49 in this term.
I20260709 12:16:35.645358 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e893ccf562714101b57af43c7f136787" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.645984 25047 leader_election.cc:304] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6483f8987428479a99e8e1c6e6be4d49; no voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1
I20260709 12:16:35.646881 25280 raft_consensus.cc:3060] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.647188 25199 leader_election.cc:304] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f; no voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.647995 25300 raft_consensus.cc:3060] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.653848 25280 raft_consensus.cc:2749] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.654415 24978 catalog_manager.cc:5697] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.656206 25300 raft_consensus.cc:2749] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.663004 25301 raft_consensus.cc:493] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.663532 25301 raft_consensus.cc:515] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.665256 25301 leader_election.cc:290] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.666164 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997f329a972472fb5bec13119cb2c52" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.666296 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997f329a972472fb5bec13119cb2c52" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.666813 25082 raft_consensus.cc:2468] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.666965 25233 raft_consensus.cc:2468] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.667835 25125 leader_election.cc:304] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.668898 25301 raft_consensus.cc:2804] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.669189 25301 raft_consensus.cc:493] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.669559 25301 raft_consensus.cc:3060] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.675451 25301 raft_consensus.cc:515] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.677052 25301 leader_election.cc:290] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.677847 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997f329a972472fb5bec13119cb2c52" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:35.677996 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d997f329a972472fb5bec13119cb2c52" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.678341 25082 raft_consensus.cc:3060] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.678638 25233 raft_consensus.cc:3060] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.683447 25233 raft_consensus.cc:2468] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:35.683768 25082 raft_consensus.cc:2468] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:35.684378 25122 leader_election.cc:304] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
W20260709 12:16:35.685329 25185 tablet.cc:2406] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:35.685243 25301 raft_consensus.cc:2804] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.685962 25301 raft_consensus.cc:697] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Becoming Leader. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:35.686869 25301 consensus_queue.cc:237] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.695050 24978 catalog_manager.cc:5697] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5888170dcea146da989a6993c1612df1 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "5888170dcea146da989a6993c1612df1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.713629 25301 raft_consensus.cc:493] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.714072 25301 raft_consensus.cc:515] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.715744 25301 leader_election.cc:290] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.716658 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.716780 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:35.717270 25082 raft_consensus.cc:2468] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.717461 25233 raft_consensus.cc:2468] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 0.
I20260709 12:16:35.718384 25125 leader_election.cc:304] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.719249 25301 raft_consensus.cc:2804] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.719645 25301 raft_consensus.cc:493] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.720036 25301 raft_consensus.cc:3060] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.726207 25301 raft_consensus.cc:515] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.728252 25301 leader_election.cc:290] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:35.729027 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:35.729656 25082 raft_consensus.cc:3060] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.729952 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:35.730522 25233 raft_consensus.cc:3060] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.737154 25082 raft_consensus.cc:2468] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:35.737213 25233 raft_consensus.cc:2468] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:35.738621 25122 leader_election.cc:304] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:35.739372 25301 raft_consensus.cc:2804] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.739895 25301 raft_consensus.cc:697] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Becoming Leader. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:35.740670 25301 consensus_queue.cc:237] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.751143 24978 catalog_manager.cc:5697] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5888170dcea146da989a6993c1612df1 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "5888170dcea146da989a6993c1612df1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.754261 25300 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.754916 25300 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.757172 25300 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.758612 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.759588 25158 raft_consensus.cc:2393] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3feccf1b43484fcfb09ad15e894e901f in current term 1: Already voted for candidate 5888170dcea146da989a6993c1612df1 in this term.
I20260709 12:16:35.758878 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.762760 25082 raft_consensus.cc:2393] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3feccf1b43484fcfb09ad15e894e901f in current term 1: Already voted for candidate 6483f8987428479a99e8e1c6e6be4d49 in this term.
I20260709 12:16:35.763828 25199 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f; no voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:35.764863 25300 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.770376 25300 raft_consensus.cc:2749] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.780166 25280 consensus_queue.cc:1048] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.797483 25302 consensus_queue.cc:1048] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.814181 25300 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.819403 25300 raft_consensus.cc:493] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.819887 25300 raft_consensus.cc:515] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.822413 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5495c5a44e6e4af0859dfb93da9c4605" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:35.822938 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5495c5a44e6e4af0859dfb93da9c4605" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:35.823196 25082 raft_consensus.cc:2468] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 0.
I20260709 12:16:35.823629 25157 raft_consensus.cc:2468] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 0.
I20260709 12:16:35.824656 25300 leader_election.cc:290] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.824604 25199 leader_election.cc:304] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.825508 25300 raft_consensus.cc:2804] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.825984 25300 raft_consensus.cc:493] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.826299 25300 raft_consensus.cc:3060] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.833264 25300 raft_consensus.cc:515] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.837383 25300 leader_election.cc:290] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:35.839710 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5495c5a44e6e4af0859dfb93da9c4605" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:35.840365 25158 raft_consensus.cc:3060] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.841737 25300 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.844360 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5495c5a44e6e4af0859dfb93da9c4605" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:35.845033 25082 raft_consensus.cc:3060] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.853840 25082 raft_consensus.cc:2468] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 1.
I20260709 12:16:35.855190 25199 leader_election.cc:304] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:35.856139 25300 raft_consensus.cc:2804] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.856755 25300 raft_consensus.cc:697] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 1 LEADER]: Becoming Leader. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:35.857731 25300 consensus_queue.cc:237] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:35.863584 25158 raft_consensus.cc:2468] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 1.
I20260709 12:16:35.871237 24978 catalog_manager.cc:5697] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3feccf1b43484fcfb09ad15e894e901f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:35.891057 25301 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.902458 25304 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.910182 25304 consensus_queue.cc:1048] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.921677 25304 consensus_queue.cc:1048] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.126332 25302 consensus_queue.cc:1048] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:36.127628 25030 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:36.138744 25331 consensus_queue.cc:1048] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.140221 25331 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.148301 25280 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:36.163013 25348 consensus_queue.cc:1048] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.171895 25304 consensus_queue.cc:1048] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.183668 25304 consensus_queue.cc:1048] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.197027 25304 consensus_queue.cc:1048] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:36.301239 25300 consensus_queue.cc:1048] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.314193 25343 consensus_queue.cc:1048] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:37.130728 25030 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:37.297159 25280 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.297832 25278 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.297816 25280 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.298441 25278 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.299804 25280 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:37.300832 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1" is_pre_election: true
I20260709 12:16:37.301110 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:37.301411 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49" is_pre_election: true
I20260709 12:16:37.301512 25157 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 1.
I20260709 12:16:37.301852 25232 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 1.
I20260709 12:16:37.302175 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f" is_pre_election: true
I20260709 12:16:37.301967 25082 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:37.301126 25278 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:37.302750 25049 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:37.302974 25233 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 1.
I20260709 12:16:37.303807 25280 raft_consensus.cc:2804] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:37.304167 25280 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.304189 25125 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:37.304833 25280 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.305320 25278 raft_consensus.cc:2804] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:37.305701 25278 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.306032 25278 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.311879 25280 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.312692 25278 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.314065 25280 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:37.315099 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:37.315789 25278 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 2 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:37.315675 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:37.315995 25157 raft_consensus.cc:2393] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6483f8987428479a99e8e1c6e6be4d49 in current term 2: Already voted for candidate 5888170dcea146da989a6993c1612df1 in this term.
I20260709 12:16:37.316548 25082 raft_consensus.cc:2393] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5888170dcea146da989a6993c1612df1 in current term 2: Already voted for candidate 6483f8987428479a99e8e1c6e6be4d49 in this term.
I20260709 12:16:37.316418 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:37.317001 25233 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.317849 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:37.319263 25122 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5888170dcea146da989a6993c1612df1; no voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:37.320057 25278 raft_consensus.cc:2749] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:37.323470 25233 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:37.324636 25047 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 5888170dcea146da989a6993c1612df1
I20260709 12:16:37.325529 25280 raft_consensus.cc:2804] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:37.326824 25280 raft_consensus.cc:697] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:37.327771 25280 consensus_queue.cc:237] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.336326 24978 catalog_manager.cc:5697] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 0 to 2, leader changed from <none> to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:37.447715 25233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
mode: GRACEFUL
new_leader_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:37544
I20260709 12:16:37.448405 25233 raft_consensus.cc:606] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 LEADER]: Received request to transfer leadership to TS 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:37.465848 25157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "5888170dcea146da989a6993c1612df1"
mode: GRACEFUL
new_leader_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:47230
I20260709 12:16:37.466372 25157 raft_consensus.cc:606] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Received request to transfer leadership to TS 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:37.485113 25083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
mode: GRACEFUL
new_leader_uuid: "5888170dcea146da989a6993c1612df1"
 from {username='slave'} at 127.0.0.1:53092
I20260709 12:16:37.485630 25083 raft_consensus.cc:606] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Received request to transfer leadership to TS 5888170dcea146da989a6993c1612df1
I20260709 12:16:37.629448 25278 raft_consensus.cc:993] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1: : Instructing follower 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:37.629906 25372 raft_consensus.cc:1081] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Signalling peer 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:37.631551 25082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:51480
I20260709 12:16:37.632071 25082 raft_consensus.cc:493] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:37.632330 25082 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.637247 25082 raft_consensus.cc:515] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.638844 25082 leader_election.cc:290] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:37.639901 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:37.640074 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:37.640551 25157 raft_consensus.cc:3055] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:37.640630 25233 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.640913 25157 raft_consensus.cc:740] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:37.641737 25157 consensus_queue.cc:260] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.642822 25157 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.645587 25233 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:37.646638 25047 leader_election.cc:304] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:37.647349 25280 raft_consensus.cc:2804] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:37.647913 25280 raft_consensus.cc:697] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:37.648905 25157 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:37.648855 25280 consensus_queue.cc:237] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.656917 24978 catalog_manager.cc:5697] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 1 to 2, leader changed from 5888170dcea146da989a6993c1612df1 (127.23.217.66) to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:37.743467 25157 raft_consensus.cc:1275] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:37.744939 25280 consensus_queue.cc:1048] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:37.755875 25233 raft_consensus.cc:1275] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:37.757469 25376 consensus_queue.cc:1048] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:37.769637 25280 raft_consensus.cc:993] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: : Instructing follower 5888170dcea146da989a6993c1612df1 to start an election
I20260709 12:16:37.770116 25362 raft_consensus.cc:1081] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Signalling peer 5888170dcea146da989a6993c1612df1 to start an election
I20260709 12:16:37.771926 25157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f"
dest_uuid: "5888170dcea146da989a6993c1612df1"
 from {username='slave'} at 127.0.0.1:32932
I20260709 12:16:37.772503 25157 raft_consensus.cc:493] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:37.772877 25157 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.779096 25157 raft_consensus.cc:515] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.780794 25157 leader_election.cc:290] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 2 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:37.781575 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:37.782271 25082 raft_consensus.cc:3055] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:37.782315 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:37.782667 25082 raft_consensus.cc:740] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:37.783085 25233 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.783470 25082 consensus_queue.cc:260] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.784485 25082 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.787891 25233 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 2.
I20260709 12:16:37.788975 25122 leader_election.cc:304] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:37.789709 25372 raft_consensus.cc:2804] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:37.789855 25082 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 2.
I20260709 12:16:37.790287 25372 raft_consensus.cc:697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 LEADER]: Becoming Leader. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:37.791208 25372 consensus_queue.cc:237] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.798381 24982 catalog_manager.cc:5697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 reported cstate change: term changed from 1 to 2, leader changed from 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65) to 5888170dcea146da989a6993c1612df1 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "5888170dcea146da989a6993c1612df1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:37.816242 25342 raft_consensus.cc:993] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: : Instructing follower 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:37.816690 25382 raft_consensus.cc:1081] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 LEADER]: Signalling peer 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:37.818185 25082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:51474
I20260709 12:16:37.818679 25082 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:37.818936 25082 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.823431 25082 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.825099 25082 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:37.825994 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:37.826248 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:37.826623 25157 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.826906 25233 raft_consensus.cc:3055] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:37.827235 25233 raft_consensus.cc:740] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:37.828047 25233 consensus_queue.cc:260] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.828956 25233 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.832239 25157 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:37.833292 25049 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:37.833693 25233 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:37.834199 25362 raft_consensus.cc:2804] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:37.834779 25362 raft_consensus.cc:697] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:37.835722 25362 consensus_queue.cc:237] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:37.846275 24982 catalog_manager.cc:5697] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 1 to 2, leader changed from 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67) to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:38.116133 25157 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:38.117676 25362 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.127141 25233 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:38.128574 25385 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.215091 25082 raft_consensus.cc:1275] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Refusing update from remote peer 5888170dcea146da989a6993c1612df1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:38.216548 25372 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.225695 25233 raft_consensus.cc:1275] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Refusing update from remote peer 5888170dcea146da989a6993c1612df1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:38.227656 25388 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.237933 25157 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:38.239550 25362 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.249135 25233 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:38.250659 25376 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:40.525149 24421 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:16:40.525811 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:40.543337 25082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
mode: GRACEFUL
new_leader_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:53098
I20260709 12:16:40.543820 25082 raft_consensus.cc:606] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Received request to transfer leadership to TS 3feccf1b43484fcfb09ad15e894e901f
I20260709 12:16:40.546190 25082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
mode: GRACEFUL
new_leader_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:53098
I20260709 12:16:40.546694 25082 raft_consensus.cc:606] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Received request to transfer leadership to TS 3feccf1b43484fcfb09ad15e894e901f
I20260709 12:16:40.547681 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:40.548053 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:40.625245 25392 raft_consensus.cc:993] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: : Instructing follower 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:40.625855 25395 raft_consensus.cc:1081] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Signalling peer 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:40.627636 25233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:38650
I20260709 12:16:40.628193 25233 raft_consensus.cc:493] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:40.628504 25233 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:40.634246 25233 raft_consensus.cc:515] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:40.635864 25233 leader_election.cc:290] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 3 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:40.637034 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:40.637081 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:40.637580 25158 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:40.637764 25082 raft_consensus.cc:3055] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:40.638151 25082 raft_consensus.cc:740] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:40.638820 25082 consensus_queue.cc:260] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:40.639875 25082 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:40.643136 25158 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 3.
I20260709 12:16:40.644347 25198 leader_election.cc:304] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:40.645172 25382 raft_consensus.cc:2804] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:40.645262 25082 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 3.
I20260709 12:16:40.646421 25382 raft_consensus.cc:697] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Becoming Leader. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:40.647280 25382 consensus_queue.cc:237] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:40.654457 24978 catalog_manager.cc:5697] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f reported cstate change: term changed from 2 to 3, leader changed from 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65) to 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67). New cstate: current_term: 3 leader_uuid: "3feccf1b43484fcfb09ad15e894e901f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:40.822775 25392 raft_consensus.cc:993] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: : Instructing follower 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:40.823249 25395 raft_consensus.cc:1081] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Signalling peer 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:40.824620 25233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:38650
I20260709 12:16:40.824985 25233 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:40.825212 25233 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:40.829684 25233 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:40.831246 25233 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 3 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:40.832216 25082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:40.832616 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:40.832856 25082 raft_consensus.cc:3055] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:40.833264 25158 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:40.833269 25082 raft_consensus.cc:740] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:40.834380 25082 consensus_queue.cc:260] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:40.835536 25082 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:40.839668 25158 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 3.
I20260709 12:16:40.840865 25198 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:40.841717 25382 raft_consensus.cc:2804] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:40.841914 25082 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 3.
I20260709 12:16:40.842186 25382 raft_consensus.cc:697] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Becoming Leader. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:40.843148 25382 consensus_queue.cc:237] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:40.850884 24978 catalog_manager.cc:5697] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f reported cstate change: term changed from 2 to 3, leader changed from 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65) to 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67). New cstate: current_term: 3 leader_uuid: "3feccf1b43484fcfb09ad15e894e901f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:41.076807 25082 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:41.078174 25382 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:41.087608 25158 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:41.089178 25382 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:41.298022 25082 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:41.299329 25382 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:41.309686 25158 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:41.312479 25396 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:41.548907 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:41.555156 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:41.555583 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:42.556524 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:42.562340 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:42.562820 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:43.564003 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:43.570147 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:43.570580 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:44.657292 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:44.662940 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:44.663517 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:16:45.090937 25105 debug-util.cc:398] Leaking SignalData structure 0x7b080014d740 after lost signal to thread 24424
W20260709 12:16:45.092060 25105 debug-util.cc:398] Leaking SignalData structure 0x7b08001ee100 after lost signal to thread 25016
W20260709 12:16:45.093076 25105 debug-util.cc:398] Leaking SignalData structure 0x7b0800058600 after lost signal to thread 25108
W20260709 12:16:45.093969 25105 debug-util.cc:398] Leaking SignalData structure 0x7b08000da340 after lost signal to thread 25183
W20260709 12:16:45.094861 25105 debug-util.cc:398] Leaking SignalData structure 0x7b08000bdf20 after lost signal to thread 25257
I20260709 12:16:45.664819 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:45.674436 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:45.675220 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:46.676432 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:46.682379 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:46.682946 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:48.130788 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:48.136389 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:48.136874 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:16:48.997541 25180 debug-util.cc:398] Leaking SignalData structure 0x7b08000fe2e0 after lost signal to thread 24424
W20260709 12:16:48.998464 25180 debug-util.cc:398] Leaking SignalData structure 0x7b08000fe9c0 after lost signal to thread 25016
W20260709 12:16:48.999595 25180 debug-util.cc:398] Leaking SignalData structure 0x7b080011b280 after lost signal to thread 25108
W20260709 12:16:49.000470 25180 debug-util.cc:398] Leaking SignalData structure 0x7b08001f2700 after lost signal to thread 25183
W20260709 12:16:49.001308 25180 debug-util.cc:398] Leaking SignalData structure 0x7b080005ae40 after lost signal to thread 25257
I20260709 12:16:49.137830 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:49.143689 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:49.144199 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:50.145342 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:50.151141 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:50.151656 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:51.192673 24421 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:16:51.201560 25233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
mode: GRACEFUL
new_leader_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:37544
I20260709 12:16:51.202559 25233 raft_consensus.cc:606] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Received request to transfer leadership to TS 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:51.211270 25233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
mode: GRACEFUL
new_leader_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:37544
I20260709 12:16:51.211820 25233 raft_consensus.cc:606] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Received request to transfer leadership to TS 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:51.219398 25158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0"
dest_uuid: "5888170dcea146da989a6993c1612df1"
mode: GRACEFUL
new_leader_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:47230
I20260709 12:16:51.219960 25158 raft_consensus.cc:606] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Received request to transfer leadership to TS 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:51.224795 25158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f"
dest_uuid: "5888170dcea146da989a6993c1612df1"
mode: GRACEFUL
new_leader_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:47230
I20260709 12:16:51.225270 25158 raft_consensus.cc:606] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 LEADER]: Received request to transfer leadership to TS 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:51.230316 25158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4"
dest_uuid: "5888170dcea146da989a6993c1612df1"
mode: GRACEFUL
new_leader_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:47230
I20260709 12:16:51.230866 25158 raft_consensus.cc:606] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Received request to transfer leadership to TS 6483f8987428479a99e8e1c6e6be4d49
I20260709 12:16:51.279301 25435 raft_consensus.cc:993] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1: : Instructing follower 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.280042 25434 raft_consensus.cc:1081] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Signalling peer 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.281934 25083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:51480
I20260709 12:16:51.282676 25083 raft_consensus.cc:493] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.283143 25083 raft_consensus.cc:3060] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.292524 25083 raft_consensus.cc:515] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.296337 25083 leader_election.cc:290] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:51.297147 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:51.297993 25158 raft_consensus.cc:3055] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:51.297919 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b3ee815a2a40a89e20379f5b91daa0" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:51.298554 25158 raft_consensus.cc:740] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:51.298815 25233 raft_consensus.cc:3060] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.299559 25158 consensus_queue.cc:260] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.300895 25158 raft_consensus.cc:3060] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.308588 25158 raft_consensus.cc:2468] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:51.310761 25049 leader_election.cc:304] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:51.312971 25433 raft_consensus.cc:2804] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:51.315089 25436 raft_consensus.cc:993] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: : Instructing follower 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.315868 25233 raft_consensus.cc:2468] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:51.315927 25432 raft_consensus.cc:1081] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Signalling peer 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.321440 25084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:51474
I20260709 12:16:51.322295 25084 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.322810 25084 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.327592 25433 raft_consensus.cc:697] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:51.328797 25433 consensus_queue.cc:237] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.331435 25440 raft_consensus.cc:993] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1: : Instructing follower 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.332144 25440 raft_consensus.cc:1081] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Signalling peer 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.334365 25083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:51480
I20260709 12:16:51.335379 25434 raft_consensus.cc:993] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1: : Instructing follower 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.335271 25083 raft_consensus.cc:493] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.335959 25434 raft_consensus.cc:1081] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 LEADER]: Signalling peer 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.336015 25083 raft_consensus.cc:3060] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.340942 25079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:51480
I20260709 12:16:51.341750 25079 raft_consensus.cc:493] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.342161 25079 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:51.345389 25083 raft_consensus.cc:515] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.348744 25158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:51.348321 24982 catalog_manager.cc:5697] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 1 to 2, leader changed from 5888170dcea146da989a6993c1612df1 (127.23.217.66) to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.349290 25158 raft_consensus.cc:3055] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:51.349579 25158 raft_consensus.cc:740] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:51.350452 25158 consensus_queue.cc:260] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.351589 25158 raft_consensus.cc:3060] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.351821 25079 raft_consensus.cc:515] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.353139 25084 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.355415 25084 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 4 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:51.355556 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:51.355723 25159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:51.356177 25233 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:51.356421 25159 raft_consensus.cc:3055] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:51.356925 25159 raft_consensus.cc:740] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:51.358790 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:51.359670 25232 raft_consensus.cc:3055] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:51.362182 25079 leader_election.cc:290] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 3 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:51.362661 25157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:51.363517 25157 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.369239 25232 raft_consensus.cc:740] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:51.370707 25232 consensus_queue.cc:260] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.395368 25233 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 3.
I20260709 12:16:51.396960 25047 leader_election.cc:304] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:51.397498 25436 raft_consensus.cc:993] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f: : Instructing follower 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.397953 25433 raft_consensus.cc:2804] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:51.398530 25433 raft_consensus.cc:697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:51.399745 25436 raft_consensus.cc:1081] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Signalling peer 6483f8987428479a99e8e1c6e6be4d49 to start an election
I20260709 12:16:51.399541 25433 consensus_queue.cc:237] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.401536 25232 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.403983 25079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
 from {username='slave'} at 127.0.0.1:51474
I20260709 12:16:51.404624 25079 raft_consensus.cc:493] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.405001 25079 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.410887 25159 consensus_queue.cc:260] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.413942 25079 raft_consensus.cc:515] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.421347 25156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:51.422633 25156 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.423377 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:51.424232 25233 raft_consensus.cc:3055] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:51.424698 25233 raft_consensus.cc:740] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:51.425864 25233 consensus_queue.cc:260] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.427234 25233 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.430676 25159 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:51.439116 25233 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 4.
I20260709 12:16:51.445595 25047 leader_election.cc:304] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:51.439572 25083 leader_election.cc:290] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:51.446875 25437 raft_consensus.cc:2804] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:51.442445 25231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377d4faa6ec445fcb31987aac4aeebe4" candidate_uuid: "6483f8987428479a99e8e1c6e6be4d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:51.448592 25231 raft_consensus.cc:3060] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.449862 25158 raft_consensus.cc:2468] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:51.451584 25049 leader_election.cc:304] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5888170dcea146da989a6993c1612df1, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:51.457690 25231 raft_consensus.cc:2468] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 2.
I20260709 12:16:51.458364 25437 raft_consensus.cc:697] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:51.458819 25445 raft_consensus.cc:2804] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:51.459743 25445 raft_consensus.cc:697] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:51.459736 25437 consensus_queue.cc:237] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.421888 25079 leader_election.cc:290] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 4 election: Requested vote from peers 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:51.465555 25232 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 4.
I20260709 12:16:51.466281 25445 consensus_queue.cc:237] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.468931 25157 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 4.
I20260709 12:16:51.469620 25047 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 6483f8987428479a99e8e1c6e6be4d49; no voters: 
I20260709 12:16:51.472762 25156 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 4.
I20260709 12:16:51.484148 25159 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6483f8987428479a99e8e1c6e6be4d49 in term 3.
I20260709 12:16:51.473229 25433 raft_consensus.cc:2804] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:51.491778 25433 raft_consensus.cc:697] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Becoming Leader. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:51.492776 25433 consensus_queue.cc:237] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:51.515025 24982 catalog_manager.cc:5697] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 3 to 4, leader changed from 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67) to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 4 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.516499 24982 catalog_manager.cc:5697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 2 to 3, leader changed from 5888170dcea146da989a6993c1612df1 (127.23.217.66) to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 3 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.517704 24982 catalog_manager.cc:5697] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 1 to 2, leader changed from 5888170dcea146da989a6993c1612df1 (127.23.217.66) to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.518908 24982 catalog_manager.cc:5697] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 reported cstate change: term changed from 3 to 4, leader changed from 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67) to 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65). New cstate: current_term: 4 leader_uuid: "6483f8987428479a99e8e1c6e6be4d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.813547 25232 raft_consensus.cc:1275] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:51.815013 25433 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:51.828080 25159 raft_consensus.cc:1275] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:51.828918 25156 raft_consensus.cc:1275] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.830011 25446 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:51.830570 25433 consensus_queue.cc:1048] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.847601 25232 raft_consensus.cc:1275] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.849005 25433 consensus_queue.cc:1048] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.866345 25156 raft_consensus.cc:1275] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.867650 25433 consensus_queue.cc:1048] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.876724 25232 raft_consensus.cc:1275] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.878825 25448 consensus_queue.cc:1048] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.893046 25232 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:51.894608 25448 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:51.896092 25156 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:51.897493 25448 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:51.908874 25159 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:51.910692 25446 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:51.928052 25231 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Refusing update from remote peer 6483f8987428479a99e8e1c6e6be4d49: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:51.929627 25433 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:54.269703 24421 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:16:54.270596 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:54.278469 25079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
mode: GRACEFUL
new_leader_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:53098
I20260709 12:16:54.279440 25079 raft_consensus.cc:606] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Received request to transfer leadership to TS 3feccf1b43484fcfb09ad15e894e901f
I20260709 12:16:54.282280 25079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
mode: GRACEFUL
new_leader_uuid: "5888170dcea146da989a6993c1612df1"
 from {username='slave'} at 127.0.0.1:53098
I20260709 12:16:54.282722 25079 raft_consensus.cc:606] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Received request to transfer leadership to TS 5888170dcea146da989a6993c1612df1
I20260709 12:16:54.285029 25079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
mode: GRACEFUL
new_leader_uuid: "5888170dcea146da989a6993c1612df1"
 from {username='slave'} at 127.0.0.1:53098
I20260709 12:16:54.285454 25079 raft_consensus.cc:606] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 LEADER]: Received request to transfer leadership to TS 5888170dcea146da989a6993c1612df1
I20260709 12:16:54.287869 25079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3"
dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
mode: GRACEFUL
new_leader_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:53098
I20260709 12:16:54.288269 25079 raft_consensus.cc:606] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Received request to transfer leadership to TS 3feccf1b43484fcfb09ad15e894e901f
I20260709 12:16:54.289350 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:54.289919 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:54.315640 25472 raft_consensus.cc:993] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: : Instructing follower 5888170dcea146da989a6993c1612df1 to start an election
I20260709 12:16:54.316247 25471 raft_consensus.cc:1081] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Signalling peer 5888170dcea146da989a6993c1612df1 to start an election
I20260709 12:16:54.317967 25159 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24"
dest_uuid: "5888170dcea146da989a6993c1612df1"
 from {username='slave'} at 127.0.0.1:32932
I20260709 12:16:54.318565 25159 raft_consensus.cc:493] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.318993 25159 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:54.326220 25159 raft_consensus.cc:515] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.328502 25159 leader_election.cc:290] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 5 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:54.329471 25079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:54.329741 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6075ea7b15354e628f157bd3c2fffb24" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:54.330556 25232 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:54.330479 25079 raft_consensus.cc:3055] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:54.331230 25079 raft_consensus.cc:740] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:54.332320 25079 consensus_queue.cc:260] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.333701 25079 raft_consensus.cc:3060] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:54.338142 25232 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 5.
I20260709 12:16:54.339897 25122 leader_election.cc:304] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:54.340979 25439 raft_consensus.cc:2804] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:54.341243 25079 raft_consensus.cc:2468] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 5.
I20260709 12:16:54.341612 25472 raft_consensus.cc:993] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: : Instructing follower 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:54.342329 25472 raft_consensus.cc:1081] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Signalling peer 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:54.342295 25439 raft_consensus.cc:697] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 5 LEADER]: Becoming Leader. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:54.343699 25439 consensus_queue.cc:237] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.344645 25231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2"
dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:38650
I20260709 12:16:54.345274 25231 raft_consensus.cc:493] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.345636 25231 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:54.353101 25231 raft_consensus.cc:515] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.353039 24982 catalog_manager.cc:5697] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 reported cstate change: term changed from 4 to 5, leader changed from 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65) to 5888170dcea146da989a6993c1612df1 (127.23.217.66). New cstate: current_term: 5 leader_uuid: "5888170dcea146da989a6993c1612df1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.356333 25231 leader_election.cc:290] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 5 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:54.356923 25079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:54.357613 25079 raft_consensus.cc:3055] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:54.357753 25159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5256fb4c554d58944c09a672b974a2" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:54.357985 25079 raft_consensus.cc:740] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:54.358325 25159 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:54.358934 25079 consensus_queue.cc:260] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.360358 25079 raft_consensus.cc:3060] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:54.366408 25159 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 5.
I20260709 12:16:54.368153 25198 leader_election.cc:304] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:54.369158 25432 raft_consensus.cc:2804] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:54.369690 25432 raft_consensus.cc:697] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 5 LEADER]: Becoming Leader. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:54.370967 25432 consensus_queue.cc:237] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.367826 25079 raft_consensus.cc:2468] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 5.
I20260709 12:16:54.384083 24978 catalog_manager.cc:5697] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f reported cstate change: term changed from 4 to 5, leader changed from 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65) to 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67). New cstate: current_term: 5 leader_uuid: "3feccf1b43484fcfb09ad15e894e901f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.451591 25471 raft_consensus.cc:993] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: : Instructing follower 5888170dcea146da989a6993c1612df1 to start an election
I20260709 12:16:54.452116 25472 raft_consensus.cc:1081] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 LEADER]: Signalling peer 5888170dcea146da989a6993c1612df1 to start an election
I20260709 12:16:54.453699 25156 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f"
dest_uuid: "5888170dcea146da989a6993c1612df1"
 from {username='slave'} at 127.0.0.1:32932
I20260709 12:16:54.454178 25156 raft_consensus.cc:493] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.454538 25156 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:54.459908 25156 raft_consensus.cc:515] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.462793 25156 leader_election.cc:290] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 4 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67:42691)
I20260709 12:16:54.462960 25079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:54.463590 25079 raft_consensus.cc:3055] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:54.463894 25231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0437decc4cc4ee1b6d0ee3fbb48a32f" candidate_uuid: "5888170dcea146da989a6993c1612df1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
I20260709 12:16:54.464612 25231 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:54.463930 25079 raft_consensus.cc:740] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:54.465580 25079 consensus_queue.cc:260] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.467176 25079 raft_consensus.cc:3060] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:54.473146 25231 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 4.
I20260709 12:16:54.474713 25122 leader_election.cc:304] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:54.475152 25079 raft_consensus.cc:2468] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5888170dcea146da989a6993c1612df1 in term 4.
I20260709 12:16:54.475755 25439 raft_consensus.cc:2804] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:54.476446 25439 raft_consensus.cc:697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 4 LEADER]: Becoming Leader. State: Replica: 5888170dcea146da989a6993c1612df1, State: Running, Role: LEADER
I20260709 12:16:54.477245 25439 consensus_queue.cc:237] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.484778 24978 catalog_manager.cc:5697] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 reported cstate change: term changed from 3 to 4, leader changed from 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65) to 5888170dcea146da989a6993c1612df1 (127.23.217.66). New cstate: current_term: 4 leader_uuid: "5888170dcea146da989a6993c1612df1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.738109 25471 raft_consensus.cc:993] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49: : Instructing follower 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:54.738657 25472 raft_consensus.cc:1081] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Signalling peer 3feccf1b43484fcfb09ad15e894e901f to start an election
I20260709 12:16:54.740291 25231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3"
dest_uuid: "3feccf1b43484fcfb09ad15e894e901f"
 from {username='slave'} at 127.0.0.1:38650
I20260709 12:16:54.740769 25231 raft_consensus.cc:493] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.741036 25231 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:54.746704 25231 raft_consensus.cc:515] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.749022 25231 leader_election.cc:290] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 3 election: Requested vote from peers 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669), 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371)
I20260709 12:16:54.750061 25079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "6483f8987428479a99e8e1c6e6be4d49"
I20260709 12:16:54.750300 25156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2a68fa2ab654a65b609f2e04a7111c3" candidate_uuid: "3feccf1b43484fcfb09ad15e894e901f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5888170dcea146da989a6993c1612df1"
I20260709 12:16:54.750725 25079 raft_consensus.cc:3055] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:54.750896 25156 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:54.751058 25079 raft_consensus.cc:740] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6483f8987428479a99e8e1c6e6be4d49, State: Running, Role: LEADER
I20260709 12:16:54.752274 25079 consensus_queue.cc:260] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.753818 25079 raft_consensus.cc:3060] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:54.757122 25156 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 3.
I20260709 12:16:54.758567 25198 leader_election.cc:304] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3feccf1b43484fcfb09ad15e894e901f, 5888170dcea146da989a6993c1612df1; no voters: 
I20260709 12:16:54.759565 25479 raft_consensus.cc:2804] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:54.759797 25079 raft_consensus.cc:2468] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3feccf1b43484fcfb09ad15e894e901f in term 3.
I20260709 12:16:54.760090 25479 raft_consensus.cc:697] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Becoming Leader. State: Replica: 3feccf1b43484fcfb09ad15e894e901f, State: Running, Role: LEADER
I20260709 12:16:54.761539 25479 consensus_queue.cc:237] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } }
I20260709 12:16:54.770119 24978 catalog_manager.cc:5697] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f reported cstate change: term changed from 2 to 3, leader changed from 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65) to 3feccf1b43484fcfb09ad15e894e901f (127.23.217.67). New cstate: current_term: 3 leader_uuid: "3feccf1b43484fcfb09ad15e894e901f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.821035 25079 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:54.823011 25479 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:16:54.830793 25231 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 5 FOLLOWER]: Refusing update from remote peer 5888170dcea146da989a6993c1612df1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:54.832218 25439 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:54.837481 25156 raft_consensus.cc:1275] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 5 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:54.839520 25480 consensus_queue.cc:1048] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:54.857200 25083 raft_consensus.cc:1275] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Refusing update from remote peer 5888170dcea146da989a6993c1612df1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:54.858747 25439 consensus_queue.cc:1048] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:55.028839 25083 raft_consensus.cc:1275] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Refusing update from remote peer 5888170dcea146da989a6993c1612df1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:55.030450 25490 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:55.041793 25231 raft_consensus.cc:1275] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Refusing update from remote peer 5888170dcea146da989a6993c1612df1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:55.044258 25490 consensus_queue.cc:1048] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3feccf1b43484fcfb09ad15e894e901f" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:55.227530 25083 raft_consensus.cc:1275] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:55.229072 25480 consensus_queue.cc:1048] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6483f8987428479a99e8e1c6e6be4d49" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 33669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.234263 25156 raft_consensus.cc:1275] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Refusing update from remote peer 3feccf1b43484fcfb09ad15e894e901f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:55.236136 25480 consensus_queue.cc:1048] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5888170dcea146da989a6993c1612df1" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.291299 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:55.296475 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:55.296852 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:56.298002 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:56.304016 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:56.304497 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:57.305517 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:57.312101 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:57.312561 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:58.313545 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:58.319320 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:58.319730 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:59.320708 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:59.326254 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:59.326807 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:00.327796 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:00.335000 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:00.335441 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:01.336561 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:01.343317 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:01.343739 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:02.345067 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:02.350366 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:02.350847 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:03.352385 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:03.359443 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:03.359854 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:04.396677 24421 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:17:04.397431 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
I20260709 12:17:04.438758 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:04.439602 24421 tablet_replica.cc:333] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.440347 24421 raft_consensus.cc:2243] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.441459 24421 raft_consensus.cc:2272] T 81b3ee815a2a40a89e20379f5b91daa0 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.443549 24421 tablet_replica.cc:333] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.444110 24421 raft_consensus.cc:2243] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.444568 24421 raft_consensus.cc:2272] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 6483f8987428479a99e8e1c6e6be4d49 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.446317 24421 tablet_replica.cc:333] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.446905 24421 raft_consensus.cc:2243] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.447342 24421 raft_consensus.cc:2272] T 5495c5a44e6e4af0859dfb93da9c4605 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.449054 24421 tablet_replica.cc:333] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.449564 24421 raft_consensus.cc:2243] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.450336 24421 raft_consensus.cc:2272] T 2e8490ab48874dccaed8b9cfc46427df P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.452106 24421 tablet_replica.cc:333] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.452605 24421 raft_consensus.cc:2243] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.453029 24421 raft_consensus.cc:2272] T 6075ea7b15354e628f157bd3c2fffb24 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.454910 24421 tablet_replica.cc:333] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.455420 24421 raft_consensus.cc:2243] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.455837 24421 raft_consensus.cc:2272] T a2a68fa2ab654a65b609f2e04a7111c3 P 6483f8987428479a99e8e1c6e6be4d49 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.457484 24421 tablet_replica.cc:333] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.457998 24421 raft_consensus.cc:2243] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.458899 24421 raft_consensus.cc:2272] T 377d4faa6ec445fcb31987aac4aeebe4 P 6483f8987428479a99e8e1c6e6be4d49 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.460523 24421 tablet_replica.cc:333] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.461027 24421 raft_consensus.cc:2243] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.461462 24421 raft_consensus.cc:2272] T 1b5256fb4c554d58944c09a672b974a2 P 6483f8987428479a99e8e1c6e6be4d49 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.463199 24421 tablet_replica.cc:333] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.463678 24421 raft_consensus.cc:2243] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.464077 24421 raft_consensus.cc:2272] T d997f329a972472fb5bec13119cb2c52 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.465718 24421 tablet_replica.cc:333] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49: stopping tablet replica
I20260709 12:17:04.466187 24421 raft_consensus.cc:2243] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.467007 24421 raft_consensus.cc:2272] T e893ccf562714101b57af43c7f136787 P 6483f8987428479a99e8e1c6e6be4d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.516189 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
W20260709 12:17:04.529583 25199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.65:33669: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260709 12:17:04.535120 25199 consensus_peers.cc:597] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f -> Peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Couldn't send request to peer 6483f8987428479a99e8e1c6e6be4d49. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:33669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.541726 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
W20260709 12:17:04.557564 25198 consensus_peers.cc:597] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f -> Peer 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371): Couldn't send request to peer 5888170dcea146da989a6993c1612df1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.576511 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:04.577168 24421 tablet_replica.cc:333] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.577920 24421 raft_consensus.cc:2243] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.578553 24421 raft_consensus.cc:2272] T 5495c5a44e6e4af0859dfb93da9c4605 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.580943 24421 tablet_replica.cc:333] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.581607 24421 raft_consensus.cc:2243] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.582084 24421 raft_consensus.cc:2272] T 2e8490ab48874dccaed8b9cfc46427df P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:04.584057 25199 consensus_peers.cc:597] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f -> Peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Couldn't send request to peer 6483f8987428479a99e8e1c6e6be4d49. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:33669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.584445 24421 tablet_replica.cc:333] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.585001 24421 raft_consensus.cc:2243] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.585423 24421 raft_consensus.cc:2272] T 81b3ee815a2a40a89e20379f5b91daa0 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.587147 24421 tablet_replica.cc:333] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.587738 24421 raft_consensus.cc:2243] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.588488 24421 raft_consensus.cc:2272] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 5888170dcea146da989a6993c1612df1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.590135 24421 tablet_replica.cc:333] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.590706 24421 raft_consensus.cc:2243] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.591435 24421 raft_consensus.cc:2272] T 6075ea7b15354e628f157bd3c2fffb24 P 5888170dcea146da989a6993c1612df1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.593145 24421 tablet_replica.cc:333] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.593667 24421 raft_consensus.cc:2243] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.594087 24421 raft_consensus.cc:2272] T a2a68fa2ab654a65b609f2e04a7111c3 P 5888170dcea146da989a6993c1612df1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.595834 24421 tablet_replica.cc:333] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.596310 24421 raft_consensus.cc:2243] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.597162 24421 raft_consensus.cc:2272] T d997f329a972472fb5bec13119cb2c52 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.598831 24421 tablet_replica.cc:333] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.599309 24421 raft_consensus.cc:2243] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.599726 24421 raft_consensus.cc:2272] T e893ccf562714101b57af43c7f136787 P 5888170dcea146da989a6993c1612df1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.601498 24421 tablet_replica.cc:333] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.602072 24421 raft_consensus.cc:2243] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.602600 24421 raft_consensus.cc:2272] T 377d4faa6ec445fcb31987aac4aeebe4 P 5888170dcea146da989a6993c1612df1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.604216 24421 tablet_replica.cc:333] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1: stopping tablet replica
I20260709 12:17:04.604682 24421 raft_consensus.cc:2243] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.605082 24421 raft_consensus.cc:2272] T 1b5256fb4c554d58944c09a672b974a2 P 5888170dcea146da989a6993c1612df1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.653291 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
W20260709 12:17:04.675621 25198 consensus_peers.cc:597] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f -> Peer 5888170dcea146da989a6993c1612df1 (127.23.217.66:37371): Couldn't send request to peer 5888170dcea146da989a6993c1612df1. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:37371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.677802 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
W20260709 12:17:04.686275 25199 consensus_peers.cc:597] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f -> Peer 6483f8987428479a99e8e1c6e6be4d49 (127.23.217.65:33669): Couldn't send request to peer 6483f8987428479a99e8e1c6e6be4d49. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:33669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.710887 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:04.711777 24421 tablet_replica.cc:333] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.712865 24421 raft_consensus.cc:2243] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.713572 24421 raft_consensus.cc:2272] T a0437decc4cc4ee1b6d0ee3fbb48a32f P 3feccf1b43484fcfb09ad15e894e901f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.715476 24421 tablet_replica.cc:333] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.715988 24421 raft_consensus.cc:2243] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.716425 24421 raft_consensus.cc:2272] T e893ccf562714101b57af43c7f136787 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.718057 24421 tablet_replica.cc:333] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.718600 24421 raft_consensus.cc:2243] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.718982 24421 raft_consensus.cc:2272] T d997f329a972472fb5bec13119cb2c52 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.720553 24421 tablet_replica.cc:333] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.721029 24421 raft_consensus.cc:2243] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.721776 24421 raft_consensus.cc:2272] T a2a68fa2ab654a65b609f2e04a7111c3 P 3feccf1b43484fcfb09ad15e894e901f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.723454 24421 tablet_replica.cc:333] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.723970 24421 raft_consensus.cc:2243] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.724399 24421 raft_consensus.cc:2272] T 6075ea7b15354e628f157bd3c2fffb24 P 3feccf1b43484fcfb09ad15e894e901f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.725981 24421 tablet_replica.cc:333] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.726466 24421 raft_consensus.cc:2243] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.727276 24421 raft_consensus.cc:2272] T 1b5256fb4c554d58944c09a672b974a2 P 3feccf1b43484fcfb09ad15e894e901f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.728919 24421 tablet_replica.cc:333] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.729389 24421 raft_consensus.cc:2243] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.729791 24421 raft_consensus.cc:2272] T 377d4faa6ec445fcb31987aac4aeebe4 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.731472 24421 tablet_replica.cc:333] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.731986 24421 raft_consensus.cc:2243] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.732756 24421 raft_consensus.cc:2272] T 5495c5a44e6e4af0859dfb93da9c4605 P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.734620 24421 tablet_replica.cc:333] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.735085 24421 raft_consensus.cc:2243] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.735484 24421 raft_consensus.cc:2272] T 2e8490ab48874dccaed8b9cfc46427df P 3feccf1b43484fcfb09ad15e894e901f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.737509 24421 tablet_replica.cc:333] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f: stopping tablet replica
I20260709 12:17:04.737941 24421 raft_consensus.cc:2243] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.738340 24421 raft_consensus.cc:2272] T 81b3ee815a2a40a89e20379f5b91daa0 P 3feccf1b43484fcfb09ad15e894e901f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.768793 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
I20260709 12:17:04.790943 24421 master.cc:562] Master@127.23.217.126:39547 shutting down...
I20260709 12:17:04.818233 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.819051 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.819419 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P 99f814c53bc7430ca37581ee70d95fb7: stopping tablet replica
I20260709 12:17:04.839246 24421 master.cc:584] Master@127.23.217.126:39547 shutdown complete.
3feccf1b43484fcfb09ad15e894e901f  1
5888170dcea146da989a6993c1612df1  4
6483f8987428479a99e8e1c6e6be4d49  5
3feccf1b43484fcfb09ad15e894e901f  3
5888170dcea146da989a6993c1612df1  4
6483f8987428479a99e8e1c6e6be4d49  3
3feccf1b43484fcfb09ad15e894e901f  1
5888170dcea146da989a6993c1612df1  1
6483f8987428479a99e8e1c6e6be4d49  8
3feccf1b43484fcfb09ad15e894e901f  3
5888170dcea146da989a6993c1612df1  3
6483f8987428479a99e8e1c6e6be4d49  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (30911 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:17:04.889763 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:36567
I20260709 12:17:04.891176 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:04.897460 25512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:04.897535 25511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:04.898227 25514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:04.900470 24421 server_base.cc:1061] running on GCE node
I20260709 12:17:04.901439 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:04.901625 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:04.901747 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599424901736 us; error 0 us; skew 500 ppm
I20260709 12:17:04.902232 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:04.905140 24421 webserver.cc:533] Webserver started at http://127.23.217.126:44231/ using document root <none> and password file <none>
I20260709 12:17:04.905607 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:04.905766 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:04.905982 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:04.907176 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "3c2fa83ead874145ba497a2c6d2a12d0"
format_stamp: "Formatted at 2026-07-09 12:17:04 on dist-test-slave-lg8f"
I20260709 12:17:04.911690 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:17:04.915198 25519 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:04.916240 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 12:17:04.916525 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "3c2fa83ead874145ba497a2c6d2a12d0"
format_stamp: "Formatted at 2026-07-09 12:17:04 on dist-test-slave-lg8f"
I20260709 12:17:04.916795 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:04.934566 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:04.935850 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:04.977175 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:36567
I20260709 12:17:04.977344 25570 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:36567 every 8 connection(s)
I20260709 12:17:04.981918 25571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:04.992990 25571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Bootstrap starting.
I20260709 12:17:04.998392 25571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:05.003597 25571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: No bootstrap required, opened a new log
I20260709 12:17:05.006227 25571 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" member_type: VOTER }
I20260709 12:17:05.006762 25571 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:05.007047 25571 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2fa83ead874145ba497a2c6d2a12d0, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.007745 25571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" member_type: VOTER }
I20260709 12:17:05.008299 25571 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:05.008538 25571 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:05.008885 25571 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:05.013747 25571 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" member_type: VOTER }
I20260709 12:17:05.014391 25571 leader_election.cc:304] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c2fa83ead874145ba497a2c6d2a12d0; no voters: 
I20260709 12:17:05.015776 25571 leader_election.cc:290] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:05.016263 25574 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:05.017747 25574 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3c2fa83ead874145ba497a2c6d2a12d0, State: Running, Role: LEADER
I20260709 12:17:05.018685 25574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" member_type: VOTER }
I20260709 12:17:05.019325 25571 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:05.024798 25576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c2fa83ead874145ba497a2c6d2a12d0. Latest consensus state: current_term: 1 leader_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" member_type: VOTER } }
I20260709 12:17:05.025537 25576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:05.025708 25575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2fa83ead874145ba497a2c6d2a12d0" member_type: VOTER } }
I20260709 12:17:05.026283 25575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:05.027765 25582 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:05.033532 25582 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:05.039925 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:17:05.044450 25582 catalog_manager.cc:1383] Generated new cluster ID: e73372a1247643b59a8e9988a7beee44
I20260709 12:17:05.044761 25582 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:05.065558 25582 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:05.067065 25582 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:05.084398 25582 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Generated new TSK 0
I20260709 12:17:05.085111 25582 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:05.109710 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:05.116510 25592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:05.117913 25593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:05.121239 25595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:05.121447 24421 server_base.cc:1061] running on GCE node
I20260709 12:17:05.122610 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:05.122838 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:05.123011 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599425122990 us; error 0 us; skew 500 ppm
I20260709 12:17:05.123544 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:05.126036 24421 webserver.cc:533] Webserver started at http://127.23.217.65:37095/ using document root <none> and password file <none>
I20260709 12:17:05.126614 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:05.126821 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:05.127097 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:05.128167 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "0ae979356d694a09be4b0cd976ea354a"
format_stamp: "Formatted at 2026-07-09 12:17:05 on dist-test-slave-lg8f"
I20260709 12:17:05.132802 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:17:05.136670 25600 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.137741 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:17:05.138029 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "0ae979356d694a09be4b0cd976ea354a"
format_stamp: "Formatted at 2026-07-09 12:17:05 on dist-test-slave-lg8f"
I20260709 12:17:05.138319 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:05.163333 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:05.164880 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:05.166476 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:05.169008 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:05.169188 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.169360 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:05.169528 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.217108 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:41483
I20260709 12:17:05.217195 25662 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:41483 every 8 connection(s)
I20260709 12:17:05.222743 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:05.231324 25667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:05.232637 25668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:05.235601 25670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:05.236943 24421 server_base.cc:1061] running on GCE node
I20260709 12:17:05.238783 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:05.239122 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:05.239370 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599425239343 us; error 0 us; skew 500 ppm
I20260709 12:17:05.240048 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:05.243367 24421 webserver.cc:533] Webserver started at http://127.23.217.66:34579/ using document root <none> and password file <none>
I20260709 12:17:05.243454 25663 heartbeater.cc:344] Connected to a master server at 127.23.217.126:36567
I20260709 12:17:05.243968 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:05.243970 25663 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:05.244477 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:05.244818 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:05.245256 25663 heartbeater.cc:507] Master 127.23.217.126:36567 requested a full tablet report, sending...
I20260709 12:17:05.246147 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "c7bf8f20701245bcbaa09cc663025f97"
format_stamp: "Formatted at 2026-07-09 12:17:05 on dist-test-slave-lg8f"
I20260709 12:17:05.247611 25536 ts_manager.cc:194] Registered new tserver with Master: 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:05.249574 25536 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42066
I20260709 12:17:05.251317 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:05.255115 25675 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.256107 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:17:05.256390 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "c7bf8f20701245bcbaa09cc663025f97"
format_stamp: "Formatted at 2026-07-09 12:17:05 on dist-test-slave-lg8f"
I20260709 12:17:05.256669 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:05.278295 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:05.279646 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:05.281213 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:05.283720 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:05.283913 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.284233 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:05.284384 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.332877 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:39947
I20260709 12:17:05.333024 25737 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:39947 every 8 connection(s)
I20260709 12:17:05.338634 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:05.346133 25741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:05.348404 25742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:05.350919 24421 server_base.cc:1061] running on GCE node
W20260709 12:17:05.351016 25744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:05.352239 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:05.352487 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:05.352692 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599425352675 us; error 0 us; skew 500 ppm
I20260709 12:17:05.353231 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:05.353518 25738 heartbeater.cc:344] Connected to a master server at 127.23.217.126:36567
I20260709 12:17:05.353945 25738 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:05.354971 25738 heartbeater.cc:507] Master 127.23.217.126:36567 requested a full tablet report, sending...
I20260709 12:17:05.356135 24421 webserver.cc:533] Webserver started at http://127.23.217.67:42619/ using document root <none> and password file <none>
I20260709 12:17:05.356742 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:05.356954 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:05.357234 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:05.357465 25536 ts_manager.cc:194] Registered new tserver with Master: c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:05.358572 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "955233c139344131a847044f17f8d348"
format_stamp: "Formatted at 2026-07-09 12:17:05 on dist-test-slave-lg8f"
I20260709 12:17:05.359110 25536 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42082
I20260709 12:17:05.363377 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:17:05.367324 25749 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.368218 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:17:05.368595 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "955233c139344131a847044f17f8d348"
format_stamp: "Formatted at 2026-07-09 12:17:05 on dist-test-slave-lg8f"
I20260709 12:17:05.368906 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:05.389099 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:05.390365 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:05.391961 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:05.394220 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:05.394404 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.394743 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:05.394913 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:05.442044 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:41061
I20260709 12:17:05.442190 25811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:41061 every 8 connection(s)
I20260709 12:17:05.457895 25812 heartbeater.cc:344] Connected to a master server at 127.23.217.126:36567
I20260709 12:17:05.458390 25812 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:05.459342 25812 heartbeater.cc:507] Master 127.23.217.126:36567 requested a full tablet report, sending...
I20260709 12:17:05.461848 25536 ts_manager.cc:194] Registered new tserver with Master: 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:05.462724 24421 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017016932s
I20260709 12:17:05.463632 25536 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42098
I20260709 12:17:05.491282 25536 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42108:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:05.498420 25536 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:05.718971 25703 tablet_service.cc:1511] Processing CreateTablet for tablet 5a760df354a14284b004635d45d88a3e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:17:05.720808 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a760df354a14284b004635d45d88a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.723922 25702 tablet_service.cc:1511] Processing CreateTablet for tablet 2a18426c9efd45e698b2b2ccb80a49e8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:17:05.725664 25702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a18426c9efd45e698b2b2ccb80a49e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.725453 25776 tablet_service.cc:1511] Processing CreateTablet for tablet 2a18426c9efd45e698b2b2ccb80a49e8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:17:05.726138 25777 tablet_service.cc:1511] Processing CreateTablet for tablet 5a760df354a14284b004635d45d88a3e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:17:05.727026 25776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a18426c9efd45e698b2b2ccb80a49e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.727779 25777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a760df354a14284b004635d45d88a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.729409 25628 tablet_service.cc:1511] Processing CreateTablet for tablet 5a760df354a14284b004635d45d88a3e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:17:05.731109 25628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a760df354a14284b004635d45d88a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.730871 25700 tablet_service.cc:1511] Processing CreateTablet for tablet 9d711fa725d244128d79dfdfb1e70af6 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:17:05.730949 25774 tablet_service.cc:1511] Processing CreateTablet for tablet 9d711fa725d244128d79dfdfb1e70af6 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:17:05.732393 25700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d711fa725d244128d79dfdfb1e70af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.732679 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d711fa725d244128d79dfdfb1e70af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.730952 25775 tablet_service.cc:1511] Processing CreateTablet for tablet b4174bd2da7548489ad8aa892dabb8bb (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:17:05.734941 25775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4174bd2da7548489ad8aa892dabb8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.736402 25626 tablet_service.cc:1511] Processing CreateTablet for tablet b4174bd2da7548489ad8aa892dabb8bb (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:17:05.737934 25626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4174bd2da7548489ad8aa892dabb8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.739921 25624 tablet_service.cc:1511] Processing CreateTablet for tablet e0c8da68209b48f8868677e30067d396 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:17:05.741451 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0c8da68209b48f8868677e30067d396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.729413 25701 tablet_service.cc:1511] Processing CreateTablet for tablet b4174bd2da7548489ad8aa892dabb8bb (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:17:05.744310 25701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4174bd2da7548489ad8aa892dabb8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.744139 25625 tablet_service.cc:1511] Processing CreateTablet for tablet 9d711fa725d244128d79dfdfb1e70af6 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:17:05.745590 25625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d711fa725d244128d79dfdfb1e70af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.747814 25699 tablet_service.cc:1511] Processing CreateTablet for tablet e0c8da68209b48f8868677e30067d396 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:17:05.751459 25627 tablet_service.cc:1511] Processing CreateTablet for tablet 2a18426c9efd45e698b2b2ccb80a49e8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:17:05.752964 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a18426c9efd45e698b2b2ccb80a49e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.755228 25699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0c8da68209b48f8868677e30067d396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.759501 25623 tablet_service.cc:1511] Processing CreateTablet for tablet 85b6713c7d844fb7a5272cac45763c0a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:17:05.761080 25623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b6713c7d844fb7a5272cac45763c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.764638 25772 tablet_service.cc:1511] Processing CreateTablet for tablet 85b6713c7d844fb7a5272cac45763c0a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:17:05.766247 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b6713c7d844fb7a5272cac45763c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.768694 25621 tablet_service.cc:1511] Processing CreateTablet for tablet 593fa615e9b64ba0974d4a40b5307c3c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:17:05.770608 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593fa615e9b64ba0974d4a40b5307c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.772316 25622 tablet_service.cc:1511] Processing CreateTablet for tablet cb52f5946670453a8ec9b21d12ececbb (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:17:05.774006 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb52f5946670453a8ec9b21d12ececbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.776717 25771 tablet_service.cc:1511] Processing CreateTablet for tablet cb52f5946670453a8ec9b21d12ececbb (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:17:05.757323 25773 tablet_service.cc:1511] Processing CreateTablet for tablet e0c8da68209b48f8868677e30067d396 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:17:05.778388 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb52f5946670453a8ec9b21d12ececbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.782192 25698 tablet_service.cc:1511] Processing CreateTablet for tablet 85b6713c7d844fb7a5272cac45763c0a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:17:05.782917 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0c8da68209b48f8868677e30067d396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.784163 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b6713c7d844fb7a5272cac45763c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.788122 25697 tablet_service.cc:1511] Processing CreateTablet for tablet cb52f5946670453a8ec9b21d12ececbb (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:17:05.789508 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb52f5946670453a8ec9b21d12ececbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.793918 25696 tablet_service.cc:1511] Processing CreateTablet for tablet 593fa615e9b64ba0974d4a40b5307c3c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:17:05.795282 25696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593fa615e9b64ba0974d4a40b5307c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.802242 25770 tablet_service.cc:1511] Processing CreateTablet for tablet 593fa615e9b64ba0974d4a40b5307c3c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:17:05.803995 25770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593fa615e9b64ba0974d4a40b5307c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.814313 25620 tablet_service.cc:1511] Processing CreateTablet for tablet 0024aee53c2a47e59a06e7e69bfb9eb2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:17:05.816855 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0024aee53c2a47e59a06e7e69bfb9eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.819092 25769 tablet_service.cc:1511] Processing CreateTablet for tablet 0024aee53c2a47e59a06e7e69bfb9eb2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:17:05.820597 25769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0024aee53c2a47e59a06e7e69bfb9eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.833837 25695 tablet_service.cc:1511] Processing CreateTablet for tablet 0024aee53c2a47e59a06e7e69bfb9eb2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:17:05.835666 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0024aee53c2a47e59a06e7e69bfb9eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.856150 25619 tablet_service.cc:1511] Processing CreateTablet for tablet 79cdf55909074ccd9e6b49679e14434d (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:17:05.863590 25619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdf55909074ccd9e6b49679e14434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.894923 25768 tablet_service.cc:1511] Processing CreateTablet for tablet 79cdf55909074ccd9e6b49679e14434d (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:17:05.895617 25698 tablet_service.cc:1511] Processing CreateTablet for tablet 79cdf55909074ccd9e6b49679e14434d (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:17:05.897162 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdf55909074ccd9e6b49679e14434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.896895 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdf55909074ccd9e6b49679e14434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.908638 25621 tablet_service.cc:1511] Processing CreateTablet for tablet afe9ac51e0a949a7a43e288f3d2ed652 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:17:05.939929 25697 tablet_service.cc:1511] Processing CreateTablet for tablet afe9ac51e0a949a7a43e288f3d2ed652 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:17:05.943367 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe9ac51e0a949a7a43e288f3d2ed652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.957926 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe9ac51e0a949a7a43e288f3d2ed652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.982762 25622 tablet_service.cc:1511] Processing CreateTablet for tablet fb1874a5bbce42ad864b2ba3624a0c1c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:17:05.985101 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb1874a5bbce42ad864b2ba3624a0c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.985643 25829 tablet_bootstrap.cc:492] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:05.990391 25702 tablet_service.cc:1511] Processing CreateTablet for tablet fb1874a5bbce42ad864b2ba3624a0c1c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:17:05.992383 25773 tablet_service.cc:1511] Processing CreateTablet for tablet afe9ac51e0a949a7a43e288f3d2ed652 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:17:05.993080 25627 tablet_service.cc:1511] Processing CreateTablet for tablet 31ccbf41d24e4275a7f921db24b940d4 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:17:05.994305 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe9ac51e0a949a7a43e288f3d2ed652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.994462 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ccbf41d24e4275a7f921db24b940d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.992887 25828 tablet_bootstrap.cc:492] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:05.992399 25702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb1874a5bbce42ad864b2ba3624a0c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.999024 25827 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:06.005733 25829 tablet_bootstrap.cc:654] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.007424 25769 tablet_service.cc:1511] Processing CreateTablet for tablet fb1874a5bbce42ad864b2ba3624a0c1c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:17:06.008785 25769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb1874a5bbce42ad864b2ba3624a0c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.019590 25703 tablet_service.cc:1511] Processing CreateTablet for tablet 31ccbf41d24e4275a7f921db24b940d4 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:17:06.021364 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ccbf41d24e4275a7f921db24b940d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.024298 25620 tablet_service.cc:1511] Processing CreateTablet for tablet e13025a828f141378027526ae4f0b002 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:17:06.025586 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13025a828f141378027526ae4f0b002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.027446 25768 tablet_service.cc:1511] Processing CreateTablet for tablet 31ccbf41d24e4275a7f921db24b940d4 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:17:06.028685 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ccbf41d24e4275a7f921db24b940d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.036506 25828 tablet_bootstrap.cc:654] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.036703 25827 tablet_bootstrap.cc:654] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:06.039817 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:06.040071 25697 tablet_service.cc:1511] Processing CreateTablet for tablet e13025a828f141378027526ae4f0b002 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:17:06.041525 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13025a828f141378027526ae4f0b002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.076967 25774 tablet_service.cc:1511] Processing CreateTablet for tablet e13025a828f141378027526ae4f0b002 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:17:06.078400 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13025a828f141378027526ae4f0b002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.086786 25698 tablet_service.cc:1511] Processing CreateTablet for tablet 571c5e0767064f3e86479b794294d327 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:17:06.088110 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571c5e0767064f3e86479b794294d327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.084832 25622 tablet_service.cc:1511] Processing CreateTablet for tablet 571c5e0767064f3e86479b794294d327 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:17:06.090004 25773 tablet_service.cc:1511] Processing CreateTablet for tablet 571c5e0767064f3e86479b794294d327 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:17:06.095733 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571c5e0767064f3e86479b794294d327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.096170 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571c5e0767064f3e86479b794294d327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.095674 25627 tablet_service.cc:1511] Processing CreateTablet for tablet b97aeee11fff4d52bb8252149577f56e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:17:06.099313 25695 tablet_service.cc:1511] Processing CreateTablet for tablet b97aeee11fff4d52bb8252149577f56e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:17:06.099809 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97aeee11fff4d52bb8252149577f56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.100696 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97aeee11fff4d52bb8252149577f56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.104296 25829 tablet_bootstrap.cc:492] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:06.104802 25829 ts_tablet_manager.cc:1403] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.120s	user 0.030s	sys 0.012s
I20260709 12:17:06.109205 25827 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:06.109710 25827 ts_tablet_manager.cc:1403] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.111s	user 0.018s	sys 0.012s
I20260709 12:17:06.113320 25776 tablet_service.cc:1511] Processing CreateTablet for tablet b97aeee11fff4d52bb8252149577f56e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:17:06.114916 25776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97aeee11fff4d52bb8252149577f56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.116350 25828 tablet_bootstrap.cc:492] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:06.116803 25828 ts_tablet_manager.cc:1403] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.124s	user 0.010s	sys 0.008s
I20260709 12:17:06.118623 25621 tablet_service.cc:1511] Processing CreateTablet for tablet c3c8e03aa6454e118cd3f9f25c785bef (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:17:06.119630 25828 raft_consensus.cc:359] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.120196 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c8e03aa6454e118cd3f9f25c785bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.122243 25828 raft_consensus.cc:385] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.122750 25828 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.123415 25827 raft_consensus.cc:359] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.123667 25828 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.124150 25827 raft_consensus.cc:385] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.125028 25827 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.126430 25827 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.132680 25697 tablet_service.cc:1511] Processing CreateTablet for tablet c3c8e03aa6454e118cd3f9f25c785bef (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:17:06.134337 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c8e03aa6454e118cd3f9f25c785bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.139436 25829 raft_consensus.cc:359] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.143023 25768 tablet_service.cc:1511] Processing CreateTablet for tablet c3c8e03aa6454e118cd3f9f25c785bef (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:17:06.145480 25829 raft_consensus.cc:385] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.145892 25829 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.146454 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c8e03aa6454e118cd3f9f25c785bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.147202 25829 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.144644 25620 tablet_service.cc:1511] Processing CreateTablet for tablet e5735f590eff4fc9ae2686d09f8b6810 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:17:06.149415 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5735f590eff4fc9ae2686d09f8b6810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.160037 25812 heartbeater.cc:499] Master 127.23.217.126:36567 was elected leader, sending a full tablet report...
I20260709 12:17:06.172263 25828 ts_tablet_manager.cc:1434] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.055s	user 0.014s	sys 0.000s
I20260709 12:17:06.179823 25828 tablet_bootstrap.cc:492] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:06.194319 25833 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.195109 25833 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:06.196878 25813 tablet.cc:2406] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:17:06.204015 25833 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:06.216321 25829 ts_tablet_manager.cc:1434] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.111s	user 0.009s	sys 0.000s
I20260709 12:17:06.220924 25663 heartbeater.cc:499] Master 127.23.217.126:36567 was elected leader, sending a full tablet report...
I20260709 12:17:06.221282 25829 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:06.234607 25698 tablet_service.cc:1511] Processing CreateTablet for tablet e5735f590eff4fc9ae2686d09f8b6810 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:17:06.236217 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5735f590eff4fc9ae2686d09f8b6810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.241794 25620 tablet_service.cc:1511] Processing CreateTablet for tablet b9052f9b88ca4b9ebb24b8764ad345af (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:17:06.246821 25738 heartbeater.cc:499] Master 127.23.217.126:36567 was elected leader, sending a full tablet report...
I20260709 12:17:06.248287 25827 ts_tablet_manager.cc:1434] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.138s	user 0.050s	sys 0.049s
I20260709 12:17:06.249408 25827 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:06.250523 25621 tablet_service.cc:1511] Processing CreateTablet for tablet b907e0d1516b4c78a2a977e29402bff9 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:17:06.251399 25828 tablet_bootstrap.cc:654] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.251735 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9052f9b88ca4b9ebb24b8764ad345af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.252425 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b907e0d1516b4c78a2a977e29402bff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.253793 25695 tablet_service.cc:1511] Processing CreateTablet for tablet b9052f9b88ca4b9ebb24b8764ad345af (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:17:06.255167 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9052f9b88ca4b9ebb24b8764ad345af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.256994 25827 tablet_bootstrap.cc:654] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.259630 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:06.261513 25752 leader_election.cc:343] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.266718 25697 tablet_service.cc:1511] Processing CreateTablet for tablet b907e0d1516b4c78a2a977e29402bff9 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:17:06.268118 25829 tablet_bootstrap.cc:654] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.268549 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b907e0d1516b4c78a2a977e29402bff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.275187 25774 tablet_service.cc:1511] Processing CreateTablet for tablet b907e0d1516b4c78a2a977e29402bff9 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:17:06.276124 25776 tablet_service.cc:1511] Processing CreateTablet for tablet e5735f590eff4fc9ae2686d09f8b6810 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:17:06.276994 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b907e0d1516b4c78a2a977e29402bff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.278016 25776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5735f590eff4fc9ae2686d09f8b6810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.280004 25773 tablet_service.cc:1511] Processing CreateTablet for tablet b9052f9b88ca4b9ebb24b8764ad345af (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:17:06.293887 25627 tablet_service.cc:1511] Processing CreateTablet for tablet eb89b7cd7f4d4662ab9d1e1f55af650c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:17:06.295398 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb89b7cd7f4d4662ab9d1e1f55af650c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.308633 25703 tablet_service.cc:1511] Processing CreateTablet for tablet eb89b7cd7f4d4662ab9d1e1f55af650c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:17:06.310040 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb89b7cd7f4d4662ab9d1e1f55af650c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.310645 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9052f9b88ca4b9ebb24b8764ad345af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.324652 25768 tablet_service.cc:1511] Processing CreateTablet for tablet eb89b7cd7f4d4662ab9d1e1f55af650c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:17:06.326186 25768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb89b7cd7f4d4662ab9d1e1f55af650c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.333621 25622 tablet_service.cc:1511] Processing CreateTablet for tablet 88f6873460634e2eb3253677a6a6b857 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:17:06.335219 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f6873460634e2eb3253677a6a6b857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.338297 25699 tablet_service.cc:1511] Processing CreateTablet for tablet ab8e959328c44aed94ce17546b6198e2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:17:06.339797 25699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e959328c44aed94ce17546b6198e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.341610 25829 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:06.342141 25829 ts_tablet_manager.cc:1403] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.121s	user 0.011s	sys 0.012s
I20260709 12:17:06.345402 25829 raft_consensus.cc:359] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.346130 25770 tablet_service.cc:1511] Processing CreateTablet for tablet ab8e959328c44aed94ce17546b6198e2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:17:06.347503 25628 tablet_service.cc:1511] Processing CreateTablet for tablet 6158b789bb894b2b9ad0c019dde5476f (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:17:06.348896 25628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6158b789bb894b2b9ad0c019dde5476f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.348682 25777 tablet_service.cc:1511] Processing CreateTablet for tablet 6158b789bb894b2b9ad0c019dde5476f (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:17:06.351516 25771 tablet_service.cc:1511] Processing CreateTablet for tablet c2f7c369dcd245eeba8804ccb208c79a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:17:06.333621 25702 tablet_service.cc:1511] Processing CreateTablet for tablet 88f6873460634e2eb3253677a6a6b857 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:17:06.353895 25702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f6873460634e2eb3253677a6a6b857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.354373 25623 tablet_service.cc:1511] Processing CreateTablet for tablet c2f7c369dcd245eeba8804ccb208c79a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:17:06.355641 25623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f7c369dcd245eeba8804ccb208c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.358698 25624 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb70fbead204d9581ca707c05f2befe (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:17:06.360097 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb70fbead204d9581ca707c05f2befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.362082 25828 tablet_bootstrap.cc:492] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:06.359738 25625 tablet_service.cc:1511] Processing CreateTablet for tablet 78bc05d6a56b407bb7507c25a641dc4e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:17:06.358712 25626 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa65389a0ed4ac1b3ec08bd217d5f22 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:17:06.363420 25625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78bc05d6a56b407bb7507c25a641dc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.335829 25769 tablet_service.cc:1511] Processing CreateTablet for tablet 88f6873460634e2eb3253677a6a6b857 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:17:06.338202 25619 tablet_service.cc:1511] Processing CreateTablet for tablet ab8e959328c44aed94ce17546b6198e2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:17:06.365512 25619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e959328c44aed94ce17546b6198e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.362677 25828 ts_tablet_manager.cc:1403] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.183s	user 0.014s	sys 0.006s
I20260709 12:17:06.369695 25626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa65389a0ed4ac1b3ec08bd217d5f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.346231 25829 raft_consensus.cc:385] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.373237 25834 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.374189 25700 tablet_service.cc:1511] Processing CreateTablet for tablet 6158b789bb894b2b9ad0c019dde5476f (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:17:06.391372 25834 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.392166 25700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6158b789bb894b2b9ad0c019dde5476f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.394109 25834 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:06.374399 25701 tablet_service.cc:1511] Processing CreateTablet for tablet c2f7c369dcd245eeba8804ccb208c79a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:17:06.374843 25772 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa65389a0ed4ac1b3ec08bd217d5f22 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:17:06.375892 25696 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa65389a0ed4ac1b3ec08bd217d5f22 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:17:06.377039 25775 tablet_service.cc:1511] Processing CreateTablet for tablet 78bc05d6a56b407bb7507c25a641dc4e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:17:06.377766 25694 tablet_service.cc:1511] Processing CreateTablet for tablet 78bc05d6a56b407bb7507c25a641dc4e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:17:06.347563 25770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e959328c44aed94ce17546b6198e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.400033 25701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f7c369dcd245eeba8804ccb208c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.386746 25627 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9b915f0d524c19b87995309e7ea872 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:17:06.404011 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f7c369dcd245eeba8804ccb208c79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.390919 25829 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.405710 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9b915f0d524c19b87995309e7ea872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.412303 25777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6158b789bb894b2b9ad0c019dde5476f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.418027 25775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78bc05d6a56b407bb7507c25a641dc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.422953 25621 tablet_service.cc:1511] Processing CreateTablet for tablet 7e41fb20a5d340ab95f58b3ccc5c680f (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:17:06.424322 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e41fb20a5d340ab95f58b3ccc5c680f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.435683 25828 raft_consensus.cc:359] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.457376 25828 raft_consensus.cc:385] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.457949 25828 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.449721 25769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f6873460634e2eb3253677a6a6b857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.467912 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa65389a0ed4ac1b3ec08bd217d5f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.465793 25696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa65389a0ed4ac1b3ec08bd217d5f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.469376 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78bc05d6a56b407bb7507c25a641dc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.474320 25774 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb70fbead204d9581ca707c05f2befe (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:17:06.476032 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb70fbead204d9581ca707c05f2befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.464540 25829 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.478610 25829 ts_tablet_manager.cc:1434] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.136s	user 0.008s	sys 0.002s
I20260709 12:17:06.479677 25829 tablet_bootstrap.cc:492] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:06.481855 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb70fbead204d9581ca707c05f2befe (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:17:06.483278 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:06.493568 25638 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:06.484241 25828 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.494997 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:06.496315 25833 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:06.496675 25833 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.496994 25833 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:06.497295 25695 tablet_service.cc:1511] Processing CreateTablet for tablet 7e41fb20a5d340ab95f58b3ccc5c680f (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:17:06.498730 25695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e41fb20a5d340ab95f58b3ccc5c680f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.499315 25828 ts_tablet_manager.cc:1434] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.132s	user 0.009s	sys 0.000s
I20260709 12:17:06.500362 25828 tablet_bootstrap.cc:492] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:06.505877 25698 tablet_service.cc:1511] Processing CreateTablet for tablet 19825207f8744cf7809ab169c0ec8973 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:17:06.486166 25827 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:06.508042 25827 ts_tablet_manager.cc:1403] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.259s	user 0.040s	sys 0.029s
I20260709 12:17:06.509603 25627 tablet_service.cc:1511] Processing CreateTablet for tablet 19825207f8744cf7809ab169c0ec8973 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:17:06.510949 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19825207f8744cf7809ab169c0ec8973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.507579 25833 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.515257 25829 tablet_bootstrap.cc:654] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.528797 25771 tablet_service.cc:1511] Processing CreateTablet for tablet 7e41fb20a5d340ab95f58b3ccc5c680f (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:17:06.530203 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e41fb20a5d340ab95f58b3ccc5c680f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.531311 25827 raft_consensus.cc:359] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.532109 25827 raft_consensus.cc:385] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.532423 25827 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.533353 25827 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.538280 25827 ts_tablet_manager.cc:1434] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.003s
I20260709 12:17:06.539439 25827 tablet_bootstrap.cc:492] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:06.544608 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:06.545234 25638 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:06.545550 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:06.547065 25752 leader_election.cc:343] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.546805 25771 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9b915f0d524c19b87995309e7ea872 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:17:06.547664 25833 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:06.548177 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9b915f0d524c19b87995309e7ea872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.498081 25703 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9b915f0d524c19b87995309e7ea872 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:17:06.507266 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19825207f8744cf7809ab169c0ec8973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.562930 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb70fbead204d9581ca707c05f2befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.564916 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9b915f0d524c19b87995309e7ea872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.593174 25773 tablet_service.cc:1511] Processing CreateTablet for tablet 19825207f8744cf7809ab169c0ec8973 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20260709 12:17:06.598948 25520 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 70717.3us
I20260709 12:17:06.603122 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:06.604750 25677 leader_election.cc:343] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.590154 25638 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:06.619527 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:06.620960 25833 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:06.602365 25699 tablet_service.cc:1511] Processing CreateTablet for tablet b74717dca3554e889066a717cfd1527a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:17:06.627537 25699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b74717dca3554e889066a717cfd1527a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.594300 25626 tablet_service.cc:1511] Processing CreateTablet for tablet b74717dca3554e889066a717cfd1527a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:17:06.628718 25702 tablet_service.cc:1511] Processing CreateTablet for tablet 7701d2ef98204ad0bedc9231fe829f69 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:17:06.629549 25828 tablet_bootstrap.cc:654] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.630352 25702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701d2ef98204ad0bedc9231fe829f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.632645 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:06.634517 25676 leader_election.cc:343] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.635190 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:06.636703 25834 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:06.640292 25772 tablet_service.cc:1511] Processing CreateTablet for tablet b74717dca3554e889066a717cfd1527a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:17:06.642410 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b74717dca3554e889066a717cfd1527a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.664876 25833 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:06.665518 25846 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.666218 25828 tablet_bootstrap.cc:492] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:06.665939 25833 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.672750 25626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b74717dca3554e889066a717cfd1527a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.673528 25828 ts_tablet_manager.cc:1403] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.173s	user 0.017s	sys 0.006s
I20260709 12:17:06.673590 25625 tablet_service.cc:1511] Processing CreateTablet for tablet 9eec1cb847cc4587be1133c287c515b2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:17:06.678797 25828 raft_consensus.cc:359] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.679476 25625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eec1cb847cc4587be1133c287c515b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.679551 25828 raft_consensus.cc:385] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.680151 25828 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.681046 25828 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.683288 25828 ts_tablet_manager.cc:1434] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:17:06.685001 25777 tablet_service.cc:1511] Processing CreateTablet for tablet 7701d2ef98204ad0bedc9231fe829f69 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:17:06.666086 25846 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.698952 25828 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:06.699496 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19825207f8744cf7809ab169c0ec8973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.720467 25627 tablet_service.cc:1511] Processing CreateTablet for tablet f6cc82bb18354fbf8395678374e211b2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:17:06.731266 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cc82bb18354fbf8395678374e211b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.722116 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:06.723362 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:06.733024 25624 tablet_service.cc:1511] Processing CreateTablet for tablet bfd362b7abdb43de96e8b4d518fbb0f9 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:17:06.733655 25777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701d2ef98204ad0bedc9231fe829f69. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:06.733950 25751 leader_election.cc:343] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:06.697785 25520 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71527.6us
I20260709 12:17:06.734346 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd362b7abdb43de96e8b4d518fbb0f9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:06.737174 25752 leader_election.cc:343] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.737861 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:06.740381 25769 tablet_service.cc:1511] Processing CreateTablet for tablet 9eec1cb847cc4587be1133c287c515b2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:17:06.744444 25769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eec1cb847cc4587be1133c287c515b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.722663 25846 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:06.726920 25620 tablet_service.cc:1511] Processing CreateTablet for tablet 7701d2ef98204ad0bedc9231fe829f69 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:17:06.770969 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 9eec1cb847cc4587be1133c287c515b2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:17:06.773217 25694 tablet_service.cc:1511] Processing CreateTablet for tablet f6cc82bb18354fbf8395678374e211b2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:17:06.778196 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eec1cb847cc4587be1133c287c515b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.780196 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701d2ef98204ad0bedc9231fe829f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.788128 25699 tablet_service.cc:1511] Processing CreateTablet for tablet bfd362b7abdb43de96e8b4d518fbb0f9 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:17:06.789678 25699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd362b7abdb43de96e8b4d518fbb0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.775214 25846 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:06.794848 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cc82bb18354fbf8395678374e211b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.808585 25829 tablet_bootstrap.cc:492] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:06.809131 25829 ts_tablet_manager.cc:1403] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.330s	user 0.030s	sys 0.031s
I20260709 12:17:06.808856 25775 tablet_service.cc:1511] Processing CreateTablet for tablet bfd362b7abdb43de96e8b4d518fbb0f9 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:17:06.810421 25775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd362b7abdb43de96e8b4d518fbb0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.812829 25619 tablet_service.cc:1511] Processing CreateTablet for tablet 402bc370028b4bc097d5c2fb4342cdfa (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:17:06.814193 25619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402bc370028b4bc097d5c2fb4342cdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.814272 25828 tablet_bootstrap.cc:654] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.817806 25622 tablet_service.cc:1511] Processing CreateTablet for tablet a741c75d1e684375b51f415fd0aa0296 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:17:06.820041 25829 raft_consensus.cc:359] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.821421 25827 tablet_bootstrap.cc:654] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.822429 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 402bc370028b4bc097d5c2fb4342cdfa (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:17:06.844285 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402bc370028b4bc097d5c2fb4342cdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.844555 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a741c75d1e684375b51f415fd0aa0296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.825311 25834 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.846757 25834 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.826674 25776 tablet_service.cc:1511] Processing CreateTablet for tablet f6cc82bb18354fbf8395678374e211b2 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:17:06.848145 25776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cc82bb18354fbf8395678374e211b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.848956 25834 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:06.850566 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:06.851193 25638 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:06.852653 25677 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:06.791671 25534 catalog_manager.cc:5697] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.854638 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:06.827440 25703 tablet_service.cc:1511] Processing CreateTablet for tablet a741c75d1e684375b51f415fd0aa0296 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:17:06.843241 25829 raft_consensus.cc:385] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.857825 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a741c75d1e684375b51f415fd0aa0296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.856154 25829 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.860647 25829 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:06.870386 25676 leader_election.cc:343] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.867900 25834 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:06.873255 25834 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.873615 25700 tablet_service.cc:1511] Processing CreateTablet for tablet 7fee45cced0b4ce18dea86254386fee0 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:17:06.874929 25700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fee45cced0b4ce18dea86254386fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.873624 25834 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:06.879385 25829 ts_tablet_manager.cc:1434] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 12:17:06.878521 25621 tablet_service.cc:1511] Processing CreateTablet for tablet b1ee3e27f67449aaab6c88508697a762 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:17:06.867731 25772 tablet_service.cc:1511] Processing CreateTablet for tablet 7fee45cced0b4ce18dea86254386fee0 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 12:17:06.882824 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 145684us
I20260709 12:17:06.886751 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1ee3e27f67449aaab6c88508697a762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.890527 25829 tablet_bootstrap.cc:492] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:06.894340 25626 tablet_service.cc:1511] Processing CreateTablet for tablet 7fee45cced0b4ce18dea86254386fee0 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:17:06.893976 25696 tablet_service.cc:1511] Processing CreateTablet for tablet b1ee3e27f67449aaab6c88508697a762 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:17:06.895325 25627 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:17:06.895576 25775 tablet_service.cc:1511] Processing CreateTablet for tablet 402bc370028b4bc097d5c2fb4342cdfa (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:17:06.896347 25696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1ee3e27f67449aaab6c88508697a762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.884663 25771 tablet_service.cc:1511] Processing CreateTablet for tablet a741c75d1e684375b51f415fd0aa0296 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:17:06.896844 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fee45cced0b4ce18dea86254386fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.897430 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a741c75d1e684375b51f415fd0aa0296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.906555 25626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fee45cced0b4ce18dea86254386fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.907684 25774 tablet_service.cc:1511] Processing CreateTablet for tablet b1ee3e27f67449aaab6c88508697a762 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:17:06.909081 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1ee3e27f67449aaab6c88508697a762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.911604 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.939105 25775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402bc370028b4bc097d5c2fb4342cdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.942907 25828 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:06.943492 25828 ts_tablet_manager.cc:1403] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.259s	user 0.015s	sys 0.010s
I20260709 12:17:06.945266 25827 tablet_bootstrap.cc:492] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:06.945813 25827 ts_tablet_manager.cc:1403] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.407s	user 0.012s	sys 0.005s
I20260709 12:17:06.946657 25828 raft_consensus.cc:359] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.947448 25828 raft_consensus.cc:385] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.947741 25828 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.948632 25828 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.948760 25827 raft_consensus.cc:359] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.949577 25827 raft_consensus.cc:385] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.949908 25827 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.930325 25829 tablet_bootstrap.cc:654] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.955076 25772 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:17:06.956321 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.959087 25834 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.959175 25827 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.960732 25828 ts_tablet_manager.cc:1434] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.006s
I20260709 12:17:06.961513 25834 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:06.961922 25828 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:06.962282 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:06.963073 25638 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:06.968065 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:06.969538 25829 tablet_bootstrap.cc:492] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
W20260709 12:17:06.969587 25676 leader_election.cc:343] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.974478 25829 ts_tablet_manager.cc:1403] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.000s
I20260709 12:17:06.975317 25638 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:06.976541 25677 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:17:06.977965 25829 raft_consensus.cc:359] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.978868 25829 raft_consensus.cc:385] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:06.979265 25829 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.980422 25829 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.982440 25829 ts_tablet_manager.cc:1434] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:17:06.983023 25828 tablet_bootstrap.cc:654] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.983680 25829 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:06.987388 25834 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:06.987910 25834 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:06.988930 25834 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:06.991390 25829 tablet_bootstrap.cc:654] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:06.993415 25698 tablet_service.cc:1511] Processing CreateTablet for tablet f80a8201dcaf4f2a9290c47595f9cf7b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:17:06.994765 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80a8201dcaf4f2a9290c47595f9cf7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.995203 25774 tablet_service.cc:1511] Processing CreateTablet for tablet f80a8201dcaf4f2a9290c47595f9cf7b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:17:06.996569 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80a8201dcaf4f2a9290c47595f9cf7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.997118 25626 tablet_service.cc:1511] Processing CreateTablet for tablet bd6ac1825a4e4ab9bb0d7d2867816cbe (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:17:06.998413 25626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6ac1825a4e4ab9bb0d7d2867816cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.002238 25622 tablet_service.cc:1511] Processing CreateTablet for tablet f80a8201dcaf4f2a9290c47595f9cf7b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:17:07.008008 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80a8201dcaf4f2a9290c47595f9cf7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.011157 25700 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:17:07.012535 25700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.012887 25771 tablet_service.cc:1511] Processing CreateTablet for tablet bd6ac1825a4e4ab9bb0d7d2867816cbe (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:17:07.014142 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6ac1825a4e4ab9bb0d7d2867816cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:06.991859 25827 ts_tablet_manager.cc:1434] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.046s	user 0.002s	sys 0.006s
I20260709 12:17:07.015758 25827 tablet_bootstrap.cc:492] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
W20260709 12:17:07.024573 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48146.7us
I20260709 12:17:06.971701 25856 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.030771 25856 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.033703 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:07.035147 25677 leader_election.cc:343] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:07.042404 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:07.049753 25827 tablet_bootstrap.cc:654] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.052131 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.051476 25535 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:07.052770 25787 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:07.054024 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:07.054841 25834 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:07.055209 25834 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:07.055537 25834 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.057137 25856 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.059054 25696 tablet_service.cc:1511] Processing CreateTablet for tablet bd6ac1825a4e4ab9bb0d7d2867816cbe (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:17:07.060299 25696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6ac1825a4e4ab9bb0d7d2867816cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.061609 25625 tablet_service.cc:1511] Processing CreateTablet for tablet 60051e5723b2429582407076a003096c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:17:07.065446 25829 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:07.065523 25625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60051e5723b2429582407076a003096c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.068544 25837 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.069072 25837 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.069257 25699 tablet_service.cc:1511] Processing CreateTablet for tablet 60051e5723b2429582407076a003096c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:17:07.069095 25829 ts_tablet_manager.cc:1403] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.086s	user 0.033s	sys 0.023s
I20260709 12:17:07.070663 25699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60051e5723b2429582407076a003096c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.077975 25775 tablet_service.cc:1511] Processing CreateTablet for tablet 60051e5723b2429582407076a003096c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:17:07.079267 25775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60051e5723b2429582407076a003096c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.084180 25837 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.085491 25829 raft_consensus.cc:359] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.086599 25829 raft_consensus.cc:385] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.087024 25829 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.093227 25829 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.098889 25829 ts_tablet_manager.cc:1434] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:17:07.104995 25698 tablet_service.cc:1511] Processing CreateTablet for tablet b06b96578363435cbde83279e86e5c86 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:17:07.086082 25834 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:07.106469 25752 consensus_peers.cc:597] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:07.122994 25698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b06b96578363435cbde83279e86e5c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.124886 25827 tablet_bootstrap.cc:492] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:07.125384 25703 tablet_service.cc:1511] Processing CreateTablet for tablet c5dd00c486ca4b45b83fda5cf38079e8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:17:07.129228 25771 tablet_service.cc:1511] Processing CreateTablet for tablet b06b96578363435cbde83279e86e5c86 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:17:07.135562 25827 ts_tablet_manager.cc:1403] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.008s
I20260709 12:17:07.136314 25771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b06b96578363435cbde83279e86e5c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.136544 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:17:07.137966 25677 leader_election.cc:343] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.138689 25834 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.134241 25829 tablet_bootstrap.cc:492] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:07.141299 25828 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:07.152853 25828 ts_tablet_manager.cc:1403] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.191s	user 0.008s	sys 0.020s
I20260709 12:17:07.155213 25776 tablet_service.cc:1511] Processing CreateTablet for tablet 3a039e6df55747e1b662ecb5d069264b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:17:07.155793 25828 raft_consensus.cc:359] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.156564 25828 raft_consensus.cc:385] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.156855 25828 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.139542 25774 tablet_service.cc:1511] Processing CreateTablet for tablet c5dd00c486ca4b45b83fda5cf38079e8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:17:07.141294 25827 raft_consensus.cc:359] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.143574 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:07.146898 25850 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:07.158373 25827 raft_consensus.cc:385] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.158581 25787 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.158758 25827 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.159320 25622 tablet_service.cc:1511] Processing CreateTablet for tablet b06b96578363435cbde83279e86e5c86 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:17:07.159547 25827 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.160737 25622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b06b96578363435cbde83279e86e5c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.156788 25776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a039e6df55747e1b662ecb5d069264b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.164203 25774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5dd00c486ca4b45b83fda5cf38079e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.147058 25699 tablet_service.cc:1511] Processing CreateTablet for tablet 294a99dd73c8449f84538cf8965a6467 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:17:07.150525 25696 tablet_service.cc:1511] Processing CreateTablet for tablet 3a039e6df55747e1b662ecb5d069264b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:17:07.167093 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5dd00c486ca4b45b83fda5cf38079e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.167826 25699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 294a99dd73c8449f84538cf8965a6467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.168790 25696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a039e6df55747e1b662ecb5d069264b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.170003 25828 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.174808 25626 tablet_service.cc:1511] Processing CreateTablet for tablet 294a99dd73c8449f84538cf8965a6467 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:17:07.175565 25621 tablet_service.cc:1511] Processing CreateTablet for tablet 3a039e6df55747e1b662ecb5d069264b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:17:07.176273 25827 ts_tablet_manager.cc:1434] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:17:07.177163 25850 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.177367 25624 tablet_service.cc:1511] Processing CreateTablet for tablet b5e8bc4d29af4139b4cb0e20fc57b10a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:17:07.177969 25827 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:07.177723 25850 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.176290 25626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 294a99dd73c8449f84538cf8965a6467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.181013 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:07.181766 25637 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:07.182940 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e8bc4d29af4139b4cb0e20fc57b10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.183274 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:07.184340 25621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a039e6df55747e1b662ecb5d069264b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.185199 25846 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:07.185595 25846 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:07.185945 25846 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.191725 25772 tablet_service.cc:1511] Processing CreateTablet for tablet 294a99dd73c8449f84538cf8965a6467 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:17:07.195063 25850 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:07.196339 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:07.204813 25700 tablet_service.cc:1511] Processing CreateTablet for tablet b5e8bc4d29af4139b4cb0e20fc57b10a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:17:07.205731 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 294a99dd73c8449f84538cf8965a6467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.206106 25700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e8bc4d29af4139b4cb0e20fc57b10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.199018 25787 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:07.200382 25625 tablet_service.cc:1511] Processing CreateTablet for tablet c5dd00c486ca4b45b83fda5cf38079e8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:17:07.215760 25769 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff8dbae494348a7be4563e3bd2982b6 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:17:07.216274 25625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5dd00c486ca4b45b83fda5cf38079e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.216403 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:07.217180 25769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff8dbae494348a7be4563e3bd2982b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.217566 25834 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:07.193847 25846 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.218076 25834 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:07.218900 25834 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.220458 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:07.221045 25637 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.223111 25846 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:07.225235 25620 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac10fecbdc4482e93572bd7ff87319e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:17:07.226846 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac10fecbdc4482e93572bd7ff87319e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.202085 25829 tablet_bootstrap.cc:654] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.196784 25828 ts_tablet_manager.cc:1434] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 12:17:07.206058 25777 tablet_service.cc:1511] Processing CreateTablet for tablet b5e8bc4d29af4139b4cb0e20fc57b10a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:17:07.232777 25828 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:07.232874 25777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e8bc4d29af4139b4cb0e20fc57b10a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:07.207342 25752 leader_election.cc:343] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.209101 25627 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff8dbae494348a7be4563e3bd2982b6 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:17:07.237012 25627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff8dbae494348a7be4563e3bd2982b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.210018 25694 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff8dbae494348a7be4563e3bd2982b6 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:17:07.239058 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff8dbae494348a7be4563e3bd2982b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.246717 25637 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:07.247937 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:07.248780 25846 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:07.249251 25846 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:07.249325 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.250154 25846 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:07.250779 25601 leader_election.cc:343] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:07.262789 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55495.2us
I20260709 12:17:07.265827 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:07.267490 25827 tablet_bootstrap.cc:654] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.269640 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac10fecbdc4482e93572bd7ff87319e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 12:17:07.267531 25752 leader_election.cc:343] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.271745 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac10fecbdc4482e93572bd7ff87319e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.279610 25828 tablet_bootstrap.cc:654] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.285895 25619 tablet_service.cc:1511] Processing CreateTablet for tablet 595303b3057546cbbfda4b84f8df8912 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:17:07.287529 25619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595303b3057546cbbfda4b84f8df8912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.288901 25535 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:07.299644 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:07.301471 25603 leader_election.cc:343] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.301937 25603 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:07.303133 25837 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:07.327133 25834 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.348059 25775 tablet_service.cc:1511] Processing CreateTablet for tablet 5ac10fecbdc4482e93572bd7ff87319e (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:17:07.349594 25775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac10fecbdc4482e93572bd7ff87319e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.352439 25773 tablet_service.cc:1511] Processing CreateTablet for tablet 595303b3057546cbbfda4b84f8df8912 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:17:07.353922 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595303b3057546cbbfda4b84f8df8912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.355852 25770 tablet_service.cc:1511] Processing CreateTablet for tablet 8c30f8fbe9ff44588b4847921bab9bd0 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:17:07.357287 25770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c30f8fbe9ff44588b4847921bab9bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.360360 25703 tablet_service.cc:1511] Processing CreateTablet for tablet 8c30f8fbe9ff44588b4847921bab9bd0 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:17:07.361999 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c30f8fbe9ff44588b4847921bab9bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.358136 25535 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:07.342686 25834 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.372789 25834 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.377535 25827 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:07.378145 25827 ts_tablet_manager.cc:1403] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.200s	user 0.021s	sys 0.017s
I20260709 12:17:07.379565 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 595303b3057546cbbfda4b84f8df8912 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:17:07.382915 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.384119 25827 raft_consensus.cc:359] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:07.384579 25676 leader_election.cc:343] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.384938 25827 raft_consensus.cc:385] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.385298 25827 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.386194 25827 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.390962 25827 ts_tablet_manager.cc:1434] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:17:07.390858 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:07.395156 25638 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:07.397004 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:17:07.398288 25834 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:07.398679 25834 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:07.399004 25834 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.413125 25834 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.415230 25834 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.416026 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:07.416616 25638 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.416661 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:07.417795 25829 tablet_bootstrap.cc:492] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:07.418337 25829 ts_tablet_manager.cc:1403] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.284s	user 0.005s	sys 0.020s
W20260709 12:17:07.418670 25676 leader_election.cc:343] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.421242 25829 raft_consensus.cc:359] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.422051 25829 raft_consensus.cc:385] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.422362 25829 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.422602 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595303b3057546cbbfda4b84f8df8912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.423369 25829 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.424093 25638 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:07.425275 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:17:07.425863 25829 ts_tablet_manager.cc:1434] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:17:07.426369 25834 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:07.427054 25834 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:07.427790 25827 tablet_bootstrap.cc:492] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:07.429086 25625 tablet_service.cc:1511] Processing CreateTablet for tablet 8c30f8fbe9ff44588b4847921bab9bd0 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:17:07.427889 25834 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.435113 25828 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:07.435315 25827 tablet_bootstrap.cc:654] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.435777 25828 ts_tablet_manager.cc:1403] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.203s	user 0.015s	sys 0.012s
I20260709 12:17:07.439087 25857 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.439678 25857 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.443434 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.446242 25857 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.446027 25620 tablet_service.cc:1511] Processing CreateTablet for tablet 36db04d76d874d328966edd6be42e2c8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:17:07.447597 25828 raft_consensus.cc:359] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.448307 25828 raft_consensus.cc:385] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.448582 25828 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.449373 25828 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.452049 25828 ts_tablet_manager.cc:1434] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:17:07.453249 25828 tablet_bootstrap.cc:492] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:07.454937 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:07.455744 25638 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:07.456959 25625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c30f8fbe9ff44588b4847921bab9bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.458010 25534 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:07.462162 25677 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:07.465258 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36db04d76d874d328966edd6be42e2c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.467149 25834 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.461295 25846 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.467839 25846 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.469574 25829 tablet_bootstrap.cc:492] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:07.471869 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:07.472452 25846 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:07.472810 25638 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:07.471987 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:07.475008 25834 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:07.475354 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:07.476166 25829 tablet_bootstrap.cc:654] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.486711 25846 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.490963 25827 tablet_bootstrap.cc:492] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:07.491525 25827 ts_tablet_manager.cc:1403] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.064s	user 0.019s	sys 0.012s
I20260709 12:17:07.492625 25850 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.493129 25850 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.495110 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 36db04d76d874d328966edd6be42e2c8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:17:07.496706 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36db04d76d874d328966edd6be42e2c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.501890 25770 tablet_service.cc:1511] Processing CreateTablet for tablet 36db04d76d874d328966edd6be42e2c8 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:17:07.503885 25770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36db04d76d874d328966edd6be42e2c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.514462 25827 raft_consensus.cc:359] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.515360 25827 raft_consensus.cc:385] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.515666 25827 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.516444 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:07.516445 25827 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.517241 25713 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:07.518669 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:07.519240 25827 ts_tablet_manager.cc:1434] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.027s	user 0.016s	sys 0.012s
I20260709 12:17:07.519677 25833 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:07.520051 25833 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:07.520376 25833 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.522475 25827 tablet_bootstrap.cc:492] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:07.522719 25846 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:07.527848 25619 tablet_service.cc:1511] Processing CreateTablet for tablet 12eaa1f2655a4cc7bde6bd409ccdf780 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:17:07.529600 25827 tablet_bootstrap.cc:654] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.529721 25619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12eaa1f2655a4cc7bde6bd409ccdf780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.532297 25834 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.532963 25834 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.536585 25834 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.537897 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.538812 25787 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:07.539914 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:07.541695 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:17:07.543147 25677 leader_election.cc:343] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.544163 25834 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:07.544605 25834 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:07.544963 25834 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.546092 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:07.547637 25751 leader_election.cc:343] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.544209 25833 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.551322 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:07.551915 25713 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.553350 25834 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.559118 25703 tablet_service.cc:1511] Processing CreateTablet for tablet 12eaa1f2655a4cc7bde6bd409ccdf780 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:17:07.561017 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12eaa1f2655a4cc7bde6bd409ccdf780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.566450 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:17:07.568333 25677 leader_election.cc:343] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.568847 25834 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.571162 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:07.571879 25787 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.572283 25850 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:07.580202 25787 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:07.580431 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:07.579977 25773 tablet_service.cc:1511] Processing CreateTablet for tablet 12eaa1f2655a4cc7bde6bd409ccdf780 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:17:07.581588 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:17:07.581943 25751 leader_election.cc:343] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.582110 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12eaa1f2655a4cc7bde6bd409ccdf780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.582950 25857 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:07.583000 25833 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:07.588229 25828 tablet_bootstrap.cc:654] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.591178 25624 tablet_service.cc:1511] Processing CreateTablet for tablet 214a36d5c7d94b8ca6fed2b3e1b9ae13 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:17:07.592624 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214a36d5c7d94b8ca6fed2b3e1b9ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.599891 25713 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:07.601737 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:07.606127 25833 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:07.606863 25833 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:07.607923 25833 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.619130 25829 tablet_bootstrap.cc:492] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:07.619808 25829 ts_tablet_manager.cc:1403] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.151s	user 0.055s	sys 0.081s
I20260709 12:17:07.619947 25882 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:07.622557 25882 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:07.622850 25694 tablet_service.cc:1511] Processing CreateTablet for tablet 214a36d5c7d94b8ca6fed2b3e1b9ae13 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:17:07.624482 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214a36d5c7d94b8ca6fed2b3e1b9ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.624231 25829 raft_consensus.cc:359] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:07.624822 25677 consensus_peers.cc:597] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:07.625119 25829 raft_consensus.cc:385] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.625592 25829 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.626695 25829 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.631788 25837 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.632277 25837 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.634454 25837 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.636439 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:07.636924 25850 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:17:07.631778 25857 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:07.637336 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.644979 25857 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.645911 25786 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:07.648232 25829 ts_tablet_manager.cc:1434] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:17:07.655562 25829 tablet_bootstrap.cc:492] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:07.665344 25827 tablet_bootstrap.cc:492] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:07.665956 25827 ts_tablet_manager.cc:1403] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.144s	user 0.033s	sys 0.013s
I20260709 12:17:07.667073 25777 tablet_service.cc:1511] Processing CreateTablet for tablet 214a36d5c7d94b8ca6fed2b3e1b9ae13 (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:17:07.668220 25829 tablet_bootstrap.cc:654] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.668459 25777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214a36d5c7d94b8ca6fed2b3e1b9ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.668516 25535 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:07.674679 25624 tablet_service.cc:1511] Processing CreateTablet for tablet 8a1f6623a51a47759e59269cfa63be4b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:17:07.676026 25624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a1f6623a51a47759e59269cfa63be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.676769 25827 raft_consensus.cc:359] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.677558 25827 raft_consensus.cc:385] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.677903 25827 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.678763 25827 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.681597 25827 ts_tablet_manager.cc:1434] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.004s
I20260709 12:17:07.686319 25827 tablet_bootstrap.cc:492] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:07.689509 25697 tablet_service.cc:1511] Processing CreateTablet for tablet 8a1f6623a51a47759e59269cfa63be4b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:17:07.691187 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a1f6623a51a47759e59269cfa63be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.702587 25772 tablet_service.cc:1511] Processing CreateTablet for tablet 8a1f6623a51a47759e59269cfa63be4b (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:17:07.704129 25772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a1f6623a51a47759e59269cfa63be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.714350 25828 tablet_bootstrap.cc:492] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:07.714941 25828 ts_tablet_manager.cc:1403] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.262s	user 0.017s	sys 0.005s
I20260709 12:17:07.717686 25603 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:07.718676 25837 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.731837 25535 catalog_manager.cc:5697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:07.736380 25619 tablet_service.cc:1511] Processing CreateTablet for tablet e905093d8adb4c6797cd77172a30843c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:17:07.737859 25619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e905093d8adb4c6797cd77172a30843c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.718281 25828 raft_consensus.cc:359] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.752540 25828 raft_consensus.cc:385] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.753273 25828 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.754773 25620 tablet_service.cc:1511] Processing CreateTablet for tablet ef4fd24cfa184aa8a8a1de8f32296e6a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:17:07.756163 25620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4fd24cfa184aa8a8a1de8f32296e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.756062 25694 tablet_service.cc:1511] Processing CreateTablet for tablet e905093d8adb4c6797cd77172a30843c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:17:07.757269 25694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e905093d8adb4c6797cd77172a30843c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.765399 25828 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.776705 25697 tablet_service.cc:1511] Processing CreateTablet for tablet ef4fd24cfa184aa8a8a1de8f32296e6a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:17:07.778057 25697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4fd24cfa184aa8a8a1de8f32296e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.774462 25827 tablet_bootstrap.cc:654] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.781569 25879 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:17:07.782708 25773 tablet_service.cc:1511] Processing CreateTablet for tablet ef4fd24cfa184aa8a8a1de8f32296e6a (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:17:07.783135 25828 ts_tablet_manager.cc:1434] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.001s
I20260709 12:17:07.784998 25828 tablet_bootstrap.cc:492] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:07.796419 25773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4fd24cfa184aa8a8a1de8f32296e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.763849 25833 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:17:07.811137 25770 tablet_service.cc:1511] Processing CreateTablet for tablet e905093d8adb4c6797cd77172a30843c (DEFAULT_TABLE table=test-workload [id=7b895a80fe4046338b656f39aee06a3c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:17:07.812700 25770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e905093d8adb4c6797cd77172a30843c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:07.817929 25857 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.832787 25837 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:07.835312 25829 tablet_bootstrap.cc:492] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:07.835928 25829 ts_tablet_manager.cc:1403] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.181s	user 0.012s	sys 0.026s
I20260709 12:17:07.845574 25829 raft_consensus.cc:359] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.847345 25829 raft_consensus.cc:385] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.847849 25829 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.849788 25829 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.818569 25857 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.850736 25837 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.853838 25837 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.854446 25829 ts_tablet_manager.cc:1434] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:17:07.855474 25829 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:07.856019 25837 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.856907 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:07.857409 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:07.858390 25603 leader_election.cc:343] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.861277 25829 tablet_bootstrap.cc:654] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:07.861487 25601 leader_election.cc:343] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:07.862025 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:07.863320 25837 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:07.877781 25857 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.877595 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:07.879357 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.881163 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:07.882261 25827 tablet_bootstrap.cc:492] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:07.882802 25827 ts_tablet_manager.cc:1403] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.197s	user 0.031s	sys 0.018s
I20260709 12:17:07.884851 25857 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:07.885845 25828 tablet_bootstrap.cc:654] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.891556 25827 raft_consensus.cc:359] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.896442 25827 raft_consensus.cc:385] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.897116 25827 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.900144 25828 tablet_bootstrap.cc:492] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:07.900646 25828 ts_tablet_manager.cc:1403] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.116s	user 0.015s	sys 0.005s
I20260709 12:17:07.900849 25829 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:07.901261 25829 ts_tablet_manager.cc:1403] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.006s
I20260709 12:17:07.903249 25828 raft_consensus.cc:359] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.903954 25828 raft_consensus.cc:385] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.903642 25829 raft_consensus.cc:359] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.904304 25829 raft_consensus.cc:385] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.904563 25829 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.905395 25829 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.908125 25829 ts_tablet_manager.cc:1434] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:07.909061 25829 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:07.910980 25828 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.911906 25828 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.916492 25828 ts_tablet_manager.cc:1434] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:17:07.917768 25828 tablet_bootstrap.cc:492] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:07.921108 25829 tablet_bootstrap.cc:654] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.898082 25827 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.929754 25828 tablet_bootstrap.cc:654] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.931583 25827 ts_tablet_manager.cc:1434] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:17:07.932569 25827 tablet_bootstrap.cc:492] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:07.943995 25827 tablet_bootstrap.cc:654] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.946085 25829 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:07.944005 25828 tablet_bootstrap.cc:492] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:07.946748 25829 ts_tablet_manager.cc:1403] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.038s	user 0.006s	sys 0.012s
I20260709 12:17:07.947139 25828 ts_tablet_manager.cc:1403] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.007s
I20260709 12:17:07.949864 25829 raft_consensus.cc:359] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.950038 25828 raft_consensus.cc:359] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.950698 25829 raft_consensus.cc:385] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.950822 25828 raft_consensus.cc:385] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.951030 25829 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.951184 25828 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.951927 25828 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.951927 25829 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.954173 25857 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:07.954698 25829 ts_tablet_manager.cc:1434] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:07.959004 25828 ts_tablet_manager.cc:1434] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:17:07.960198 25828 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:07.961341 25829 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:07.967929 25829 tablet_bootstrap.cc:654] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.968722 25827 tablet_bootstrap.cc:492] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:07.969188 25827 ts_tablet_manager.cc:1403] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.007s
I20260709 12:17:07.972054 25827 raft_consensus.cc:359] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.972769 25827 raft_consensus.cc:385] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:07.973104 25827 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.973886 25787 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:07.973929 25827 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.976611 25827 ts_tablet_manager.cc:1434] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:07.977715 25827 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:07.979229 25896 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.979835 25896 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:07.984712 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:07.985438 25713 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:07.994480 25896 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:07.995944 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:07.996835 25828 tablet_bootstrap.cc:654] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:07.998322 25601 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:07.999673 25896 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.001417 25882 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.002791 25827 tablet_bootstrap.cc:654] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.006876 25828 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.007509 25828 ts_tablet_manager.cc:1403] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.003s
I20260709 12:17:08.010569 25828 raft_consensus.cc:359] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.012391 25828 raft_consensus.cc:385] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.012827 25828 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.013947 25896 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:08.015691 25829 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:08.016824 25829 ts_tablet_manager.cc:1403] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.016s
I20260709 12:17:08.016601 25828 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.019336 25828 ts_tablet_manager.cc:1434] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:17:08.019780 25829 raft_consensus.cc:359] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.020573 25828 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:08.025266 25827 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.025820 25827 ts_tablet_manager.cc:1403] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.006s
I20260709 12:17:08.027817 25828 tablet_bootstrap.cc:654] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.020537 25829 raft_consensus.cc:385] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.029124 25829 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.028913 25827 raft_consensus.cc:359] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.029835 25827 raft_consensus.cc:385] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.030186 25827 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.030206 25829 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.031358 25827 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.037701 25638 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.040693 25828 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.040999 25829 ts_tablet_manager.cc:1434] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:17:08.041421 25828 ts_tablet_manager.cc:1403] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 12:17:08.041862 25833 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.042069 25829 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:08.042445 25833 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.044588 25828 raft_consensus.cc:359] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.045491 25828 raft_consensus.cc:385] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.045856 25828 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.046550 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.047260 25847 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.047362 25637 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:08.047861 25847 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.048399 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:08.049059 25713 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:08.049191 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:17:08.049571 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:08.050246 25847 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:08.050725 25846 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.051105 25846 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.051451 25846 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.054685 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:08.055334 25713 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:08.056556 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.046243 25833 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:08.057811 25833 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.058144 25833 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.058473 25833 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.059552 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.060300 25637 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:08.061306 25827 ts_tablet_manager.cc:1434] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.004s
I20260709 12:17:08.065713 25882 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:08.066706 25829 tablet_bootstrap.cc:654] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.067390 25827 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.068614 25833 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.074679 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:08.075536 25637 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.075798 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:08.076573 25713 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.078051 25827 tablet_bootstrap.cc:654] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.081099 25828 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.086716 25713 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:08.063459 25846 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.092044 25828 ts_tablet_manager.cc:1434] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.000s
I20260709 12:17:08.092058 25833 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:08.092974 25880 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:08.093497 25828 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:08.094005 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.095472 25847 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.096611 25847 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:08.096737 25846 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:08.098543 25847 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.103385 25828 tablet_bootstrap.cc:654] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.090386 25637 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:08.109973 25829 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:08.110428 25829 ts_tablet_manager.cc:1403] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.069s	user 0.015s	sys 0.000s
I20260709 12:17:08.108702 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:08.108728 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:08.111776 25635 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.111842 25713 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.122479 25829 raft_consensus.cc:359] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.123374 25829 raft_consensus.cc:385] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.123746 25829 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.123481 25833 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.124511 25829 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.126817 25829 ts_tablet_manager.cc:1434] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:17:08.127880 25829 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:08.150705 25713 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:08.154824 25713 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.156935 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.158147 25846 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.158886 25846 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:08.159958 25846 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.178851 25635 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:08.179517 25828 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.180138 25828 ts_tablet_manager.cc:1403] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.012s
I20260709 12:17:08.183611 25828 raft_consensus.cc:359] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.184412 25828 raft_consensus.cc:385] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.184805 25828 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.185570 25828 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.186830 25827 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.187412 25827 ts_tablet_manager.cc:1403] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.120s	user 0.018s	sys 0.013s
I20260709 12:17:08.188182 25828 ts_tablet_manager.cc:1434] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:08.189203 25828 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:08.189599 25829 tablet_bootstrap.cc:654] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.190728 25827 raft_consensus.cc:359] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.191447 25827 raft_consensus.cc:385] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.191771 25827 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.195103 25827 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.204725 25827 ts_tablet_manager.cc:1434] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:17:08.205704 25827 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.206214 25833 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.213783 25536 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:08.223582 25827 tablet_bootstrap.cc:654] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.244871 25828 tablet_bootstrap.cc:654] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.250735 25536 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:08.252019 25829 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:08.252754 25829 ts_tablet_manager.cc:1403] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.125s	user 0.016s	sys 0.013s
I20260709 12:17:08.256583 25829 raft_consensus.cc:359] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.257586 25829 raft_consensus.cc:385] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.257967 25829 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.258842 25829 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.259342 25880 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.259922 25880 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.261479 25828 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.261982 25828 ts_tablet_manager.cc:1403] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.004s
I20260709 12:17:08.261997 25829 ts_tablet_manager.cc:1434] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:17:08.262207 25880 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.263309 25829 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:08.263815 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.274811 25787 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:08.276103 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.276942 25880 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.277311 25880 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.277403 25827 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.277701 25880 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.278064 25827 ts_tablet_manager.cc:1403] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.073s	user 0.009s	sys 0.004s
I20260709 12:17:08.280682 25829 tablet_bootstrap.cc:654] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.281450 25827 raft_consensus.cc:359] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.282254 25827 raft_consensus.cc:385] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.265007 25828 raft_consensus.cc:359] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.267261 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.282945 25828 raft_consensus.cc:385] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.283324 25828 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.282580 25827 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.283221 25638 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:08.284302 25828 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.271742 25896 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.286698 25896 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.289259 25828 ts_tablet_manager.cc:1434] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 12:17:08.290171 25828 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:08.292141 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.293047 25787 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:08.294076 25896 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.294551 25601 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:08.295125 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:08.297554 25603 leader_election.cc:343] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.297964 25880 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.299837 25896 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.299598 25827 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.300262 25896 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.300765 25896 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.303018 25827 ts_tablet_manager.cc:1434] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.006s
I20260709 12:17:08.300686 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:08.303656 25638 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.304379 25827 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.310683 25638 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:08.311650 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:08.322974 25787 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.311961 25833 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.323551 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.323740 25833 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.324544 25882 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.325079 25882 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:08.325881 25833 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:08.325965 25882 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.327684 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.328096 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:08.328323 25635 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:08.328666 25712 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:08.312420 25880 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.314026 25828 tablet_bootstrap.cc:654] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.314936 25827 tablet_bootstrap.cc:654] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.318775 25829 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:08.333557 25787 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:08.321300 25896 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.329578 25751 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:08.334300 25829 ts_tablet_manager.cc:1403] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.071s	user 0.017s	sys 0.026s
I20260709 12:17:08.338716 25829 raft_consensus.cc:359] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.339650 25833 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.340330 25833 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.340838 25833 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.353077 25896 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.339689 25829 raft_consensus.cc:385] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.353668 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:08.354034 25829 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
W20260709 12:17:08.355089 25603 leader_election.cc:343] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.355225 25833 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.354413 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:08.356539 25787 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.358785 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:08.359473 25638 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.362389 25829 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.364428 25828 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.364897 25828 ts_tablet_manager.cc:1403] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.075s	user 0.020s	sys 0.016s
I20260709 12:17:08.366266 25829 ts_tablet_manager.cc:1434] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 12:17:08.368167 25787 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:08.369174 25829 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:08.372951 25638 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:08.369026 25828 raft_consensus.cc:359] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.373831 25828 raft_consensus.cc:385] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.374570 25828 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.373819 25601 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:08.375355 25828 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.376256 25896 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.376672 25829 tablet_bootstrap.cc:654] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.376809 25896 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:08.374048 25535 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:08.381832 25828 ts_tablet_manager.cc:1434] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 12:17:08.383486 25828 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:08.377748 25896 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.391919 25751 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:08.393108 25918 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.393817 25918 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:08.395823 25827 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.396175 25833 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:08.396430 25827 ts_tablet_manager.cc:1403] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.018s
I20260709 12:17:08.398386 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:08.399226 25712 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.400179 25827 raft_consensus.cc:359] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.401304 25827 raft_consensus.cc:385] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.395712 25918 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.404500 25829 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:08.405036 25829 ts_tablet_manager.cc:1403] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.021s
I20260709 12:17:08.407197 25827 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.407812 25829 raft_consensus.cc:359] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.408052 25827 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.408511 25829 raft_consensus.cc:385] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.408764 25829 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.409490 25829 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.410564 25534 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:08.411897 25829 ts_tablet_manager.cc:1434] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:17:08.412245 25827 ts_tablet_manager.cc:1434] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 12:17:08.412858 25829 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:08.413113 25827 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.416052 25828 tablet_bootstrap.cc:654] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.418275 25896 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.419526 25896 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.421447 25827 tablet_bootstrap.cc:654] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.421656 25896 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.423130 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.421511 25829 tablet_bootstrap.cc:654] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:08.424554 25601 leader_election.cc:343] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.425063 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:08.426644 25603 leader_election.cc:343] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.427562 25603 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:08.427764 25712 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:08.428684 25896 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:08.438674 25827 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.439322 25827 ts_tablet_manager.cc:1403] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.007s
I20260709 12:17:08.445194 25827 raft_consensus.cc:359] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.446107 25827 raft_consensus.cc:385] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.446424 25827 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.447738 25827 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.450619 25827 ts_tablet_manager.cc:1434] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:17:08.451853 25827 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.453678 25535 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:08.468122 25827 tablet_bootstrap.cc:654] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.473325 25828 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.473841 25828 ts_tablet_manager.cc:1403] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.091s	user 0.009s	sys 0.038s
I20260709 12:17:08.476643 25828 raft_consensus.cc:359] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.477337 25828 raft_consensus.cc:385] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.477620 25828 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.478408 25828 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.480178 25829 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:08.480262 25827 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.480451 25828 ts_tablet_manager.cc:1434] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:17:08.480775 25827 ts_tablet_manager.cc:1403] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.001s
I20260709 12:17:08.480775 25829 ts_tablet_manager.cc:1403] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.068s	user 0.027s	sys 0.026s
I20260709 12:17:08.481703 25828 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:08.483785 25829 raft_consensus.cc:359] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.483897 25827 raft_consensus.cc:359] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.484499 25829 raft_consensus.cc:385] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.485019 25896 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.485059 25829 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.485581 25896 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.486033 25829 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.487718 25896 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.489084 25828 tablet_bootstrap.cc:654] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.489311 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:08.489939 25712 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:08.491550 25829 ts_tablet_manager.cc:1434] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:17:08.491390 25603 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.492259 25896 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.492412 25829 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:08.492619 25896 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.492929 25896 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.497311 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.498016 25787 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:08.484650 25827 raft_consensus.cc:385] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.499516 25827 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.500239 25827 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.498394 25829 tablet_bootstrap.cc:654] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.502298 25827 ts_tablet_manager.cc:1434] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 12:17:08.503321 25896 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.507120 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:08.507850 25712 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.517418 25712 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:08.518918 25603 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.520167 25894 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.527042 25827 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.531258 25828 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.531869 25828 ts_tablet_manager.cc:1403] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.020s
I20260709 12:17:08.536041 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:08.536734 25787 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.539178 25827 tablet_bootstrap.cc:654] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.534793 25896 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.543197 25896 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.543826 25896 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.547573 25896 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.548429 25882 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.548946 25882 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.550660 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:08.551687 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.551555 25712 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:08.552356 25638 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:08.553511 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.553857 25603 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.554818 25879 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.555186 25879 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.555512 25879 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.555558 25882 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.557374 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.545189 25894 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:08.559672 25894 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.562147 25896 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.568030 25896 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.558303 25785 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:08.564584 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.568457 25896 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.572079 25918 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:08.572324 25828 raft_consensus.cc:359] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.573719 25828 raft_consensus.cc:385] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.574108 25828 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.574958 25828 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.575981 25786 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:08.577332 25828 ts_tablet_manager.cc:1434] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:17:08.579007 25828 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:08.578862 25879 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.589705 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:08.590615 25635 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.590709 25787 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:08.591985 25829 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:08.592466 25829 ts_tablet_manager.cc:1403] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.100s	user 0.032s	sys 0.032s
I20260709 12:17:08.595908 25829 raft_consensus.cc:359] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.596935 25829 raft_consensus.cc:385] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.597308 25829 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.599867 25635 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:08.598011 25829 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.595894 25828 tablet_bootstrap.cc:654] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.600848 25833 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.603583 25936 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.605240 25936 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.604749 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:08.611105 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:08.611428 25827 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.611996 25827 ts_tablet_manager.cc:1403] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.006s
W20260709 12:17:08.614965 25603 leader_election.cc:343] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.612053 25882 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.616693 25882 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:08.617805 25882 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.626974 25936 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.625852 25827 raft_consensus.cc:359] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.628441 25827 raft_consensus.cc:385] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.628794 25827 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.634785 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.636564 25827 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.602177 25896 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.636157 25846 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:17:08.643606 25601 leader_election.cc:343] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.644368 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:08.647130 25827 ts_tablet_manager.cc:1434] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 12:17:08.647753 25894 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:08.648623 25827 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.654691 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:08.655072 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:08.655581 25787 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.655936 25712 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.660249 25879 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.652873 25896 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.663486 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:08.665457 25786 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.676762 25534 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:08.678406 25786 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:08.678332 25787 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:08.686812 25829 ts_tablet_manager.cc:1434] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.094s	user 0.024s	sys 0.022s
I20260709 12:17:08.688235 25829 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:08.692428 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:08.697198 25940 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.698698 25940 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:08.699923 25940 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.710810 25881 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.711551 25881 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.719532 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.715458 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.714458 25881 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.748979 25787 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:08.750428 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:08.753769 25881 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.759948 25828 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:08.760579 25829 tablet_bootstrap.cc:654] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.766641 25828 ts_tablet_manager.cc:1403] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.188s	user 0.036s	sys 0.025s
I20260709 12:17:08.768230 25712 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:08.761893 25534 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:08.783957 25879 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.791091 25535 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:08.792802 25940 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.784452 25882 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.798275 25940 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.799558 25882 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.786027 25846 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:17:08.809222 25882 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.811514 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:08.812018 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.812310 25786 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
W20260709 12:17:08.814024 25677 leader_election.cc:343] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.810643 25828 raft_consensus.cc:359] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.815420 25828 raft_consensus.cc:385] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.815698 25828 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.819204 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:08.820545 25882 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.821007 25882 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.821411 25882 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.826020 25828 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.830674 25827 tablet_bootstrap.cc:654] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.837471 25940 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.833768 25881 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:08.837602 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:08.839718 25828 ts_tablet_manager.cc:1434] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.001s
I20260709 12:17:08.840895 25828 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Bootstrap starting.
W20260709 12:17:08.840664 25603 leader_election.cc:343] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.842046 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:08.843653 25601 leader_election.cc:343] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.844118 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:08.845209 25940 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:08.853929 25828 tablet_bootstrap.cc:654] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.857965 25882 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.861764 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:17:08.863435 25677 leader_election.cc:343] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:08.865950 25882 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:08.866201 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:08.866804 25786 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.876235 25786 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:08.877851 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:08.879400 25881 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:08.879907 25881 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:08.880822 25881 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.893193 25827 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.893932 25827 ts_tablet_manager.cc:1403] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.246s	user 0.016s	sys 0.002s
I20260709 12:17:08.896292 25534 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:08.897358 25827 raft_consensus.cc:359] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.912415 25827 raft_consensus.cc:385] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:08.913931 25827 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:08.916235 25827 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.918314 25896 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.923233 25827 ts_tablet_manager.cc:1434] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 12:17:08.929071 25827 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:08.946194 25846 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.947094 25846 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:08.951674 25881 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.952028 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:08.952696 25638 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:08.951674 25940 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:08.956311 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:08.957712 25833 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:08.958160 25833 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.958532 25833 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:08.956578 25827 tablet_bootstrap.cc:654] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:08.976677 25827 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:08.991237 25847 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:08.996730 25827 ts_tablet_manager.cc:1403] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.068s	user 0.008s	sys 0.008s
I20260709 12:17:09.002063 25847 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:09.008090 25827 raft_consensus.cc:359] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.008981 25940 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:09.011060 25827 raft_consensus.cc:385] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.012724 25827 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.014458 25833 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.069397 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:09.070006 25638 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.014679 25827 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.072661 25827 ts_tablet_manager.cc:1434] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.001s
I20260709 12:17:09.073863 25827 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.075269 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:09.076293 25710 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.070045 25833 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.041374 25881 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:17:09.079759 25896 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:09.080614 25638 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:09.082119 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:09.086663 25918 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.091105 25918 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:09.091990 25918 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.052109 25846 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.052461 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.099339 25882 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:09.102772 25710 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:09.108696 25827 tablet_bootstrap.cc:654] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:09.122473 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:09.130695 25896 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:09.141314 25827 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.156311 25936 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:09.159988 25829 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.160472 25829 ts_tablet_manager.cc:1403] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.473s	user 0.117s	sys 0.151s
I20260709 12:17:09.141934 25827 ts_tablet_manager.cc:1403] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.000s
I20260709 12:17:09.163293 25829 raft_consensus.cc:359] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.164036 25829 raft_consensus.cc:385] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.164378 25829 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.165343 25829 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.167392 25827 raft_consensus.cc:359] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.168411 25829 ts_tablet_manager.cc:1434] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:17:09.169126 25827 raft_consensus.cc:385] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.169772 25829 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.169703 25827 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.171309 25827 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.178136 25829 tablet_bootstrap.cc:654] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.178455 25828 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.179032 25828 ts_tablet_manager.cc:1403] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.338s	user 0.064s	sys 0.108s
I20260709 12:17:09.181267 25827 ts_tablet_manager.cc:1434] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:17:09.182531 25827 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.187784 25828 raft_consensus.cc:359] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.190563 25827 tablet_bootstrap.cc:654] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.199379 25828 raft_consensus.cc:385] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.199890 25828 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.199344 25536 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:09.200770 25828 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.203392 25828 ts_tablet_manager.cc:1434] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:17:09.204355 25828 tablet_bootstrap.cc:492] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.210466 25828 tablet_bootstrap.cc:654] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.252331 25829 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.252816 25829 ts_tablet_manager.cc:1403] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.083s	user 0.017s	sys 0.001s
I20260709 12:17:09.253911 25827 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.254443 25827 ts_tablet_manager.cc:1403] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.004s
I20260709 12:17:09.254977 25828 tablet_bootstrap.cc:492] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.255426 25828 ts_tablet_manager.cc:1403] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.008s
I20260709 12:17:09.257395 25827 raft_consensus.cc:359] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.258141 25827 raft_consensus.cc:385] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.258459 25828 raft_consensus.cc:359] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.255439 25829 raft_consensus.cc:359] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.259327 25828 raft_consensus.cc:385] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.259403 25829 raft_consensus.cc:385] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.259624 25828 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.259701 25829 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.260330 25829 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.258435 25827 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.262895 25829 ts_tablet_manager.cc:1434] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:17:09.264060 25829 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.263985 25827 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.260380 25828 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.266803 25827 ts_tablet_manager.cc:1434] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:17:09.267491 25828 ts_tablet_manager.cc:1434] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:17:09.267887 25827 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.271624 25828 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.273854 25827 tablet_bootstrap.cc:654] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.275130 25881 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.275624 25881 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.277721 25881 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.278808 25829 tablet_bootstrap.cc:654] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.280260 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:09.277951 25828 tablet_bootstrap.cc:654] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.282418 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:09.284281 25676 leader_election.cc:343] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:09.285071 25677 leader_election.cc:343] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.285516 25677 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:09.286592 25882 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.287025 25882 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.289120 25882 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.289609 25882 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:09.290463 25918 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.291033 25918 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.292065 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:09.292719 25784 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:09.293056 25918 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.295778 25828 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.296264 25828 ts_tablet_manager.cc:1403] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.000s
I20260709 12:17:09.296187 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.296841 25712 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:09.298882 25828 raft_consensus.cc:359] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.299808 25828 raft_consensus.cc:385] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.300102 25828 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.300958 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:09.300879 25828 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.301580 25636 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:09.302994 25828 ts_tablet_manager.cc:1434] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:09.303563 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:09.304147 25638 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:09.295778 25829 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.305120 25829 ts_tablet_manager.cc:1403] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.003s
I20260709 12:17:09.305387 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:09.306638 25828 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.307608 25829 raft_consensus.cc:359] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.308219 25829 raft_consensus.cc:385] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.308490 25829 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.310362 25829 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.312803 25829 ts_tablet_manager.cc:1434] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:09.313799 25829 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.313874 25828 tablet_bootstrap.cc:654] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.317922 25880 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:09.318868 25880 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.319319 25880 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.322052 25829 tablet_bootstrap.cc:654] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.323208 25827 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.298990 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:09.323678 25827 ts_tablet_manager.cc:1403] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.015s
I20260709 12:17:09.324906 25918 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:09.325294 25918 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.325632 25918 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.326782 25827 raft_consensus.cc:359] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.327658 25827 raft_consensus.cc:385] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.327939 25827 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.328692 25827 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.330739 25827 ts_tablet_manager.cc:1434] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:09.331596 25827 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.347436 25846 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.347999 25846 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.350107 25846 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.351099 25827 tablet_bootstrap.cc:654] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.355906 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:09.358498 25751 leader_election.cc:343] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.360064 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:09.361711 25752 leader_election.cc:343] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.362177 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:09.362468 25827 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.362952 25827 ts_tablet_manager.cc:1403] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.008s
I20260709 12:17:09.364785 25846 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:09.368273 25827 raft_consensus.cc:359] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.368980 25827 raft_consensus.cc:385] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.369258 25827 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.369882 25827 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.371749 25827 ts_tablet_manager.cc:1434] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:17:09.377424 25638 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.379768 25918 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.382368 25882 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.382537 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:09.382885 25882 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.383311 25636 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.384893 25882 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.386610 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:09.386760 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:09.388212 25676 leader_election.cc:343] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.388670 25827 tablet_bootstrap.cc:492] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
W20260709 12:17:09.388700 25677 leader_election.cc:343] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.389488 25677 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:09.391067 25881 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:09.390987 25636 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:09.394937 25827 tablet_bootstrap.cc:654] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.394738 25751 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:09.397876 25881 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:09.399071 25846 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.399724 25846 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:09.400816 25846 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.408828 25879 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:09.415831 25880 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.419977 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:09.421396 25638 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:09.424892 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:09.426183 25712 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:09.428504 25918 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.454912 25881 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.455662 25881 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.457104 25880 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.458532 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:09.459668 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:09.460079 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:09.460608 25636 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:09.461673 25880 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:09.461846 25936 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.462543 25936 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.466077 25936 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.467585 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:09.466356 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.468824 25712 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:09.469532 25881 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.470259 25881 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:09.470692 25881 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.471134 25881 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.470989 25603 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:09.473985 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:09.476356 25676 leader_election.cc:343] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.477906 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:09.479912 25601 leader_election.cc:343] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.488683 25881 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.490263 25936 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:09.490907 25936 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.491392 25936 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.496198 25829 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.496919 25829 ts_tablet_manager.cc:1403] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.183s	user 0.040s	sys 0.036s
I20260709 12:17:09.500542 25829 raft_consensus.cc:359] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.500957 25827 tablet_bootstrap.cc:492] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.501289 25829 raft_consensus.cc:385] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.501650 25829 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.501667 25827 ts_tablet_manager.cc:1403] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.113s	user 0.043s	sys 0.055s
I20260709 12:17:09.501868 25828 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.502609 25828 ts_tablet_manager.cc:1403] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.196s	user 0.066s	sys 0.061s
I20260709 12:17:09.502655 25829 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.503584 25535 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:09.504693 25827 raft_consensus.cc:359] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.505401 25827 raft_consensus.cc:385] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.505614 25828 raft_consensus.cc:359] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.505702 25827 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.507326 25828 raft_consensus.cc:385] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.507764 25828 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.508009 25827 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.508548 25828 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.510665 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:17:09.512033 25676 leader_election.cc:343] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.512957 25936 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.513957 25918 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.514654 25918 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.518296 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:09.518687 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.518965 25712 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.519609 25828 ts_tablet_manager.cc:1434] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.009s
I20260709 12:17:09.519881 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:09.520412 25636 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.520527 25827 ts_tablet_manager.cc:1434] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:17:09.520984 25828 tablet_bootstrap.cc:492] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.522050 25846 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:09.522790 25752 leader_election.cc:343] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.522567 25846 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.523430 25918 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.524716 25846 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.525418 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:09.525995 25827 tablet_bootstrap.cc:492] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.526985 25712 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:09.527403 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:09.528287 25603 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:09.529908 25894 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.530406 25894 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:17:09.531536 25751 leader_election.cc:343] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.531284 25894 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.531989 25751 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:09.532317 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:09.533272 25827 tablet_bootstrap.cc:654] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:09.533690 25601 leader_election.cc:343] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.534386 25879 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.534922 25879 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.535835 25846 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:17:09.536542 25751 leader_election.cc:343] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.537120 25879 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.538806 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:09.540179 25677 leader_election.cc:343] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.544559 25828 tablet_bootstrap.cc:654] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.546530 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:09.548260 25636 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:09.549603 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:17:09.550554 25880 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.551087 25880 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:09.551373 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.552153 25712 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:09.551898 25880 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:09.553503 25676 leader_election.cc:343] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.553496 25752 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:09.555152 25829 ts_tablet_manager.cc:1434] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 12:17:09.555737 25846 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:09.553999 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:09.556258 25846 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.556754 25846 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.557190 25879 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:09.561126 25829 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.562901 25881 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.564579 25936 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.568684 25846 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.572763 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:09.570984 25846 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:09.573568 25712 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.575191 25828 tablet_bootstrap.cc:492] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.606212 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:09.608389 25828 ts_tablet_manager.cc:1403] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.088s	user 0.017s	sys 0.014s
I20260709 12:17:09.612879 25827 tablet_bootstrap.cc:492] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.613409 25827 ts_tablet_manager.cc:1403] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.088s	user 0.009s	sys 0.008s
I20260709 12:17:09.614674 25535 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:09.617739 25829 tablet_bootstrap.cc:654] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.621759 25827 raft_consensus.cc:359] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.622536 25827 raft_consensus.cc:385] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.622831 25827 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
W20260709 12:17:09.623371 25751 leader_election.cc:343] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.628101 25960 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:09.630009 25827 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.632985 25827 ts_tablet_manager.cc:1434] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:17:09.634012 25827 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.663805 25532 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:09.642611 25712 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:09.688652 25828 raft_consensus.cc:359] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.689870 25828 raft_consensus.cc:385] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.676162 25918 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:17:09.691141 25752 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:09.692045 25846 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.692510 25846 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:09.693305 25846 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.690286 25828 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.709436 25828 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.713580 25827 tablet_bootstrap.cc:654] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.719719 25828 ts_tablet_manager.cc:1434] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.111s	user 0.006s	sys 0.003s
I20260709 12:17:09.721421 25829 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.721958 25829 ts_tablet_manager.cc:1403] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.161s	user 0.051s	sys 0.024s
I20260709 12:17:09.724764 25829 raft_consensus.cc:359] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.725437 25829 raft_consensus.cc:385] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.725682 25829 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.727651 25828 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.726270 25829 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.734508 25829 ts_tablet_manager.cc:1434] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 12:17:09.735525 25829 tablet_bootstrap.cc:492] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.740000 25535 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:09.755285 25828 tablet_bootstrap.cc:654] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.755295 25829 tablet_bootstrap.cc:654] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.760294 25827 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.760878 25827 ts_tablet_manager.cc:1403] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.127s	user 0.018s	sys 0.008s
I20260709 12:17:09.766263 25829 tablet_bootstrap.cc:492] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.766808 25829 ts_tablet_manager.cc:1403] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.003s
I20260709 12:17:09.769227 25829 raft_consensus.cc:359] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.769755 25829 raft_consensus.cc:385] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.769994 25829 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.770663 25829 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.772346 25829 ts_tablet_manager.cc:1434] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 12:17:09.773113 25829 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.776357 25827 raft_consensus.cc:359] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.777180 25827 raft_consensus.cc:385] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.777520 25827 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.778364 25827 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.779395 25829 tablet_bootstrap.cc:654] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.781067 25827 ts_tablet_manager.cc:1434] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:17:09.782986 25827 tablet_bootstrap.cc:492] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.785036 25828 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.790882 25828 ts_tablet_manager.cc:1403] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.009s
I20260709 12:17:09.793511 25828 raft_consensus.cc:359] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.794188 25828 raft_consensus.cc:385] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.794457 25828 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.795463 25828 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.801867 25828 ts_tablet_manager.cc:1434] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:17:09.802776 25828 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.808156 25827 tablet_bootstrap.cc:654] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.811066 25829 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.811617 25829 ts_tablet_manager.cc:1403] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.039s	user 0.031s	sys 0.006s
I20260709 12:17:09.808346 25828 tablet_bootstrap.cc:654] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.814750 25829 raft_consensus.cc:359] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.815541 25829 raft_consensus.cc:385] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.815821 25829 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.816434 25829 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.817381 25936 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.817843 25936 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.820111 25936 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.822214 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.823141 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:09.823909 25827 tablet_bootstrap.cc:492] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:09.824499 25827 ts_tablet_manager.cc:1403] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.002s
W20260709 12:17:09.824822 25603 leader_election.cc:343] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.826586 25829 ts_tablet_manager.cc:1434] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 12:17:09.823819 25787 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:09.828024 25829 tablet_bootstrap.cc:492] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.828059 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:09.829396 25936 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:09.829811 25936 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.830154 25936 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.834800 25829 tablet_bootstrap.cc:654] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.837054 25936 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.840950 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:09.842672 25827 raft_consensus.cc:359] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:09.843148 25603 leader_election.cc:343] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.843410 25827 raft_consensus.cc:385] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.845358 25827 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.845639 25828 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.846187 25828 ts_tablet_manager.cc:1403] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.009s
I20260709 12:17:09.846097 25827 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.847451 25829 tablet_bootstrap.cc:492] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.847944 25829 ts_tablet_manager.cc:1403] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 12:17:09.848318 25827 ts_tablet_manager.cc:1434] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:17:09.849769 25936 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.849274 25828 raft_consensus.cc:359] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.850975 25828 raft_consensus.cc:385] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.851109 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:09.850797 25829 raft_consensus.cc:359] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.851505 25828 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.851796 25787 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.852058 25829 raft_consensus.cc:385] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.852578 25828 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.853371 25829 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.855224 25828 ts_tablet_manager.cc:1434] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:17:09.854667 25829 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.856240 25828 tablet_bootstrap.cc:492] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.858533 25829 ts_tablet_manager.cc:1434] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:17:09.858863 25880 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.859652 25829 tablet_bootstrap.cc:492] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.862799 25828 tablet_bootstrap.cc:654] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.865376 25829 tablet_bootstrap.cc:654] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.859400 25880 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.869700 25880 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.871467 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:09.874400 25676 leader_election.cc:343] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.874673 25829 tablet_bootstrap.cc:492] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.875491 25829 ts_tablet_manager.cc:1403] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.005s
I20260709 12:17:09.875651 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:09.876394 25635 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:09.877748 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:17:09.879594 25829 raft_consensus.cc:359] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.881174 25829 raft_consensus.cc:385] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.881587 25829 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.881919 25880 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:09.882333 25880 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.882822 25936 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.881021 25787 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:09.882293 25829 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.883378 25936 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.883333 25828 tablet_bootstrap.cc:492] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.885411 25828 ts_tablet_manager.cc:1403] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.010s
I20260709 12:17:09.885828 25829 ts_tablet_manager.cc:1434] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:17:09.886435 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.888130 25828 raft_consensus.cc:359] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.888842 25828 raft_consensus.cc:385] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.889142 25828 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
W20260709 12:17:09.889197 25603 leader_election.cc:343] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.890198 25828 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.890741 25936 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.886740 25829 tablet_bootstrap.cc:492] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.849303 25827 tablet_bootstrap.cc:492] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:09.891010 25960 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:09.893244 25828 ts_tablet_manager.cc:1434] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:09.894412 25828 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.882973 25880 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.896938 25827 tablet_bootstrap.cc:654] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.897131 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:09.900177 25828 tablet_bootstrap.cc:654] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.906613 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:09.908057 25601 leader_election.cc:343] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.908569 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:09.909775 25894 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.910296 25894 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:09.911232 25894 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.912104 25828 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.912673 25828 ts_tablet_manager.cc:1403] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 12:17:09.914693 25940 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:09.916079 25829 tablet_bootstrap.cc:654] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.916040 25828 raft_consensus.cc:359] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.917032 25828 raft_consensus.cc:385] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.917376 25828 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.918226 25828 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.921098 25828 ts_tablet_manager.cc:1434] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:17:09.922266 25828 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.928053 25829 tablet_bootstrap.cc:492] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:09.928499 25829 ts_tablet_manager.cc:1403] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.001s
I20260709 12:17:09.930148 25880 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.933719 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:09.934408 25635 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.935475 25936 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.935985 25936 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.954694 25881 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.955211 25828 tablet_bootstrap.cc:654] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.955273 25881 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.958431 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:09.960058 25677 leader_election.cc:343] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.960253 25829 raft_consensus.cc:359] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.961443 25829 raft_consensus.cc:385] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.961789 25829 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.966569 25829 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.968652 25829 ts_tablet_manager.cc:1434] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.040s	user 0.002s	sys 0.007s
I20260709 12:17:09.968907 25828 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:09.969357 25828 ts_tablet_manager.cc:1403] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.006s
I20260709 12:17:09.969537 25829 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:09.971524 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:09.972020 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:09.972430 25710 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:09.972959 25828 raft_consensus.cc:359] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:09.973554 25676 leader_election.cc:343] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:09.974009 25828 raft_consensus.cc:385] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.974594 25881 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.974658 25828 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.974066 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:09.975822 25828 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:09.977247 25784 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.978076 25828 ts_tablet_manager.cc:1434] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:17:09.980291 25828 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:09.983772 25882 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:09.984169 25829 tablet_bootstrap.cc:654] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.986284 25936 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:09.986416 25882 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:09.987658 25828 tablet_bootstrap.cc:654] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.998893 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:10.000402 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.001260 25940 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.004457 25896 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:10.004223 25960 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:10.009814 25940 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.023821 25880 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.026758 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:10.027411 25784 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.044636 25827 tablet_bootstrap.cc:492] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:10.045311 25827 ts_tablet_manager.cc:1403] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.196s	user 0.028s	sys 0.012s
I20260709 12:17:10.053467 25635 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:10.055233 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.055540 25784 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:10.056205 25881 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.057039 25881 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:10.058018 25881 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.060077 25829 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:10.060612 25829 ts_tablet_manager.cc:1403] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.023s
I20260709 12:17:10.067215 25532 catalog_manager.cc:5697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:10.092775 25829 raft_consensus.cc:359] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.093838 25829 raft_consensus.cc:385] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.094177 25829 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.095013 25829 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.097514 25829 ts_tablet_manager.cc:1434] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.000s
I20260709 12:17:10.100435 25828 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:10.100953 25828 ts_tablet_manager.cc:1403] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.122s	user 0.026s	sys 0.011s
I20260709 12:17:10.101539 25829 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:10.103468 25828 raft_consensus.cc:359] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.104715 25828 raft_consensus.cc:385] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.105039 25828 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.105710 25828 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.097276 25827 raft_consensus.cc:359] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.107493 25827 raft_consensus.cc:385] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.107841 25827 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
W20260709 12:17:10.108503 25601 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:10.108692 25918 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.109169 25918 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.111733 25918 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.113287 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.113170 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:10.114935 25752 leader_election.cc:343] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:10.115782 25751 leader_election.cc:343] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.116250 25751 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.116531 25960 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.117074 25960 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.117511 25918 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.108536 25827 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.117585 25828 ts_tablet_manager.cc:1434] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.016s	user 0.001s	sys 0.005s
I20260709 12:17:10.118369 25880 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.119006 25880 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.123211 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.123832 25710 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:10.124377 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.124385 25882 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:10.126688 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.126984 25880 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.127154 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.128382 25828 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:10.129699 25894 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.130210 25894 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.132326 25894 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.133631 25960 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.134315 25960 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.134356 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.134840 25960 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.135308 25960 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.136924 25827 ts_tablet_manager.cc:1434] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.091s	user 0.004s	sys 0.012s
I20260709 12:17:10.260234 25829 tablet_bootstrap.cc:654] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.137136 25918 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.141191 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:10.256209 25828 tablet_bootstrap.cc:700] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Time spent opening tablet: real 0.109s	user 0.001s	sys 0.000s
W20260709 12:17:10.146682 25603 leader_election.cc:343] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:10.258817 25751 leader_election.cc:343] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.263731 25828 tablet_bootstrap.cc:654] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.265141 25827 tablet_bootstrap.cc:492] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:10.278549 25894 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:10.265128 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 118522us
I20260709 12:17:10.262697 25918 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.279338 25894 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:10.266188 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 118371us
I20260709 12:17:10.289122 25936 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:10.286568 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 120056us
I20260709 12:17:10.310066 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:10.303596 25936 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.311038 25787 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:10.312563 25936 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.282716 26013 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:17:10.314347 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.315191 25710 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:10.315599 25960 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.316378 25894 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.317456 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.319370 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:10.319993 25601 leader_election.cc:343] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.320006 25710 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:10.320474 25601 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.321522 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.322410 25940 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.322819 25940 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.323199 25940 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.324617 25846 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.325088 25846 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.326812 25894 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.330020 25827 tablet_bootstrap.cc:654] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.339164 25918 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.341702 25828 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:10.341472 25532 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:10.342347 25828 ts_tablet_manager.cc:1403] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.214s	user 0.014s	sys 0.005s
I20260709 12:17:10.335546 25894 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.351205 25894 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.354861 25846 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.365165 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:10.366008 25712 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.360214 25960 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.368829 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:10.365255 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.371409 26011 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.361223 25894 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.377832 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.379107 25829 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:10.379767 25829 ts_tablet_manager.cc:1403] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.281s	user 0.028s	sys 0.024s
I20260709 12:17:10.378528 25711 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:10.409411 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.410409 25634 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:10.412173 25896 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.412781 25896 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.422533 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:10.452353 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:10.453336 25785 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:10.456205 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:10.456883 25785 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
W20260709 12:17:10.459457 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:10.462136 25828 raft_consensus.cc:359] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.464092 25829 raft_consensus.cc:359] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.465068 25828 raft_consensus.cc:385] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.465250 25829 raft_consensus.cc:385] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.465732 25828 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.465807 25829 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.469609 25829 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.470335 25828 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.472649 25828 ts_tablet_manager.cc:1434] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.130s	user 0.006s	sys 0.000s
W20260709 12:17:10.460415 25752 leader_election.cc:343] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.473563 25828 tablet_bootstrap.cc:492] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:10.473982 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.476130 25918 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.476544 25918 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.476858 25918 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.477813 25940 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.479359 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.488395 25829 ts_tablet_manager.cc:1434] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.108s	user 0.012s	sys 0.000s
I20260709 12:17:10.489714 25829 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:10.486719 25846 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.490849 25846 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.491217 25846 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.493094 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.493755 25709 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:10.488019 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:10.496018 25711 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.500204 25940 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.501024 25603 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.490631 25896 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.503571 25936 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.506251 25936 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.506978 25936 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.512341 25712 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:10.516893 25711 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:10.501646 25638 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.518051 25936 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.518464 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.519582 25940 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:10.519763 25894 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.525684 25896 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.525903 25881 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:10.520354 25936 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.527258 25896 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:10.531915 25918 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.532505 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.532472 25896 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.525332 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:10.536401 26020 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.536605 25894 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.536960 26020 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:10.537530 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:10.538192 25712 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.537856 26020 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.516929 25846 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.540398 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.544448 25635 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:10.540910 25711 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.547758 25827 tablet_bootstrap.cc:492] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:10.548218 25827 ts_tablet_manager.cc:1403] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.283s	user 0.024s	sys 0.006s
I20260709 12:17:10.552227 25638 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:10.554569 25712 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:10.555755 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.556241 25711 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:10.559715 25829 tablet_bootstrap.cc:654] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.566731 26018 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.567234 26018 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.565562 25603 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.571854 25894 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.572530 26018 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.573243 25827 raft_consensus.cc:359] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.572614 25894 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:10.574097 25827 raft_consensus.cc:385] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.575544 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:10.576911 25752 leader_election.cc:343] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.577467 25827 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.580741 25827 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.581585 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.582885 25827 ts_tablet_manager.cc:1434] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 12:17:10.583861 25827 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:10.585796 26011 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.587037 26023 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.587847 26023 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:10.588153 25918 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.589212 26023 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.595073 25940 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:10.598315 25846 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.598711 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:10.603050 25752 leader_election.cc:343] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.603253 25894 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.606297 25828 tablet_bootstrap.cc:654] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.604779 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:10.608474 25712 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:10.614084 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:10.615619 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:10.617650 25783 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.638427 25829 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:10.638993 25829 ts_tablet_manager.cc:1403] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.149s	user 0.012s	sys 0.002s
I20260709 12:17:10.637583 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:10.649539 25882 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:10.652624 25601 leader_election.cc:343] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:10.653836 25601 leader_election.cc:343] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.654029 25829 raft_consensus.cc:359] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.654686 25829 raft_consensus.cc:385] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.654990 25829 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.655627 25829 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.652691 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:10.659770 25786 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:10.660904 25846 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:10.662714 25751 leader_election.cc:343] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.663261 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.664060 25918 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.657790 25535 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:10.667840 25829 ts_tablet_manager.cc:1434] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 12:17:10.675583 25827 tablet_bootstrap.cc:654] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.680681 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:10.682574 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:10.683195 25636 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.689291 25536 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:10.690064 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.691918 25536 catalog_manager.cc:5697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:10.692693 25846 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.695348 25829 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:10.702077 25536 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:10.707903 25783 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:10.733530 25636 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:10.750021 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.750525 25828 tablet_bootstrap.cc:492] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:10.750981 25828 ts_tablet_manager.cc:1403] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.278s	user 0.032s	sys 0.019s
I20260709 12:17:10.751296 25918 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.751834 25918 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:10.745795 25829 tablet_bootstrap.cc:654] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.755393 25918 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.761957 25828 raft_consensus.cc:359] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.763727 25828 raft_consensus.cc:385] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.764056 25828 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.764779 25828 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.772470 25828 ts_tablet_manager.cc:1434] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:17:10.773578 25828 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:10.779482 26020 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.779743 25828 tablet_bootstrap.cc:654] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.780076 26020 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.783792 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.784533 25712 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:10.785936 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.787388 26018 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.787520 25827 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:10.787804 26018 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.788193 26018 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.788219 25827 ts_tablet_manager.cc:1403] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.205s	user 0.020s	sys 0.017s
I20260709 12:17:10.798041 25829 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:10.798710 25829 ts_tablet_manager.cc:1403] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.000s
I20260709 12:17:10.799274 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:10.803583 25751 leader_election.cc:343] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.807286 25828 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:10.807802 25828 ts_tablet_manager.cc:1403] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.008s
I20260709 12:17:10.782326 26020 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.810603 25828 raft_consensus.cc:359] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.811388 25828 raft_consensus.cc:385] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.811653 25828 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.812318 25828 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.802260 25829 raft_consensus.cc:359] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.813753 25829 raft_consensus.cc:385] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.814010 25829 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.814934 25828 ts_tablet_manager.cc:1434] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:10.814699 25829 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.815948 25828 tablet_bootstrap.cc:492] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:10.816911 25829 ts_tablet_manager.cc:1434] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:17:10.817809 25829 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:10.819710 25827 raft_consensus.cc:359] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.820375 25827 raft_consensus.cc:385] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.820701 25827 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.823575 25828 tablet_bootstrap.cc:654] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.826418 25829 tablet_bootstrap.cc:654] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.828445 25827 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.831517 25827 ts_tablet_manager.cc:1434] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 12:17:10.832885 25827 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:10.835095 25828 tablet_bootstrap.cc:492] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:10.835537 25828 ts_tablet_manager.cc:1403] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 12:17:10.838037 25828 raft_consensus.cc:359] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.838682 25828 raft_consensus.cc:385] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.838929 25828 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.839704 25828 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.842043 25828 ts_tablet_manager.cc:1434] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:17:10.842979 25828 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:10.845532 25827 tablet_bootstrap.cc:654] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.848186 26020 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.849097 25828 tablet_bootstrap.cc:654] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.848909 26020 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.855509 25829 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:10.856141 25829 ts_tablet_manager.cc:1403] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.000s
I20260709 12:17:10.854152 26020 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.859609 25829 raft_consensus.cc:359] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.860566 25829 raft_consensus.cc:385] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.860924 25829 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.861127 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:10.862983 25829 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.869026 25828 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:10.869637 25828 ts_tablet_manager.cc:1403] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.003s
I20260709 12:17:10.872357 25828 raft_consensus.cc:359] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.874943 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.874253 25829 ts_tablet_manager.cc:1434] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 12:17:10.877923 25829 tablet_bootstrap.cc:492] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:10.886319 25829 tablet_bootstrap.cc:654] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.873155 25828 raft_consensus.cc:385] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.889832 25828 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.890811 25828 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.891877 25712 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:10.893728 25828 ts_tablet_manager.cc:1434] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:17:10.894873 25828 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:10.901567 25828 tablet_bootstrap.cc:654] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.894804 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:10.913918 26018 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.915192 26017 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.918790 25829 tablet_bootstrap.cc:492] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:10.919474 25829 ts_tablet_manager.cc:1403] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.042s	user 0.006s	sys 0.008s
I20260709 12:17:10.922947 25829 raft_consensus.cc:359] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.923862 25829 raft_consensus.cc:385] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.924202 25829 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.924461 26017 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.925063 25829 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.925633 25532 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:10.929828 25828 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:10.938530 25828 ts_tablet_manager.cc:1403] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.005s
I20260709 12:17:10.939148 26018 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:10.940286 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:10.941370 25828 raft_consensus.cc:359] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.942188 25828 raft_consensus.cc:385] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.940426 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:10.942458 25828 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.941340 25712 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.943177 25828 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.936489 25827 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:10.944865 25827 ts_tablet_manager.cc:1403] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.112s	user 0.040s	sys 0.031s
I20260709 12:17:10.945230 25828 ts_tablet_manager.cc:1434] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:10.946189 25828 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:10.948011 25827 raft_consensus.cc:359] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.948792 25827 raft_consensus.cc:385] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.949122 25827 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.949388 25829 ts_tablet_manager.cc:1434] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.003s
I20260709 12:17:10.950030 25827 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.950363 25936 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:10.952653 25827 ts_tablet_manager.cc:1434] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:10.950313 25829 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:10.948022 25882 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.957257 25882 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.960553 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:10.942802 25633 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.961316 25638 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:10.966226 25829 tablet_bootstrap.cc:654] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.967482 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:10.968092 25882 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:10.969099 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:10.970978 25828 tablet_bootstrap.cc:654] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.972829 25882 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.973469 25827 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:10.973735 25936 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:10.980820 25829 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:10.981302 25829 ts_tablet_manager.cc:1403] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 12:17:10.982180 25827 tablet_bootstrap.cc:654] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.984010 25829 raft_consensus.cc:359] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.984687 25829 raft_consensus.cc:385] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.984989 25829 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.986271 25829 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:10.990545 25712 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:10.996488 25829 ts_tablet_manager.cc:1434] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:17:10.996675 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:10.997655 26017 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.997870 25829 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:10.998382 26017 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:10.999396 26017 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.002223 25882 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.004240 25633 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:11.013536 26018 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.014705 25829 tablet_bootstrap.cc:654] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.025535 25828 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:11.032528 26020 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.032887 25936 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.026201 25828 ts_tablet_manager.cc:1403] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.000s
I20260709 12:17:11.034101 25936 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.034592 25936 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.036691 25936 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.039011 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.039819 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.040328 25881 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.040467 25786 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:11.040814 25881 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.044466 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:17:11.045223 25603 leader_election.cc:343] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.046135 25894 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.046537 25894 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.046864 25894 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.037947 25828 raft_consensus.cc:359] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.047842 25828 raft_consensus.cc:385] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.048216 25828 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.051393 25829 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:11.051970 25829 ts_tablet_manager.cc:1403] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.006s
I20260709 12:17:11.055203 25829 raft_consensus.cc:359] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.055088 25828 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.056234 25829 raft_consensus.cc:385] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.056681 25829 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.057229 25881 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.057422 25829 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.059130 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:11.060985 25677 leader_election.cc:343] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.062444 25828 ts_tablet_manager.cc:1434] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 12:17:11.064916 25829 ts_tablet_manager.cc:1434] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:17:11.065828 25829 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:11.071907 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.072319 25827 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:11.072590 25786 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:11.073004 25827 ts_tablet_manager.cc:1403] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.100s	user 0.031s	sys 0.057s
I20260709 12:17:11.073302 25829 tablet_bootstrap.cc:654] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.074127 26023 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.063858 25828 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:11.074785 26023 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.077033 26023 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.078715 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.080403 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:11.080789 25786 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:11.082028 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:11.082050 25881 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.082898 26023 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.083586 26023 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.083926 26023 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.085457 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.082865 25881 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.086541 25881 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.091459 26023 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.082315 25827 raft_consensus.cc:359] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.093954 25827 raft_consensus.cc:385] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.094321 25827 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.095265 25828 tablet_bootstrap.cc:654] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.095074 25827 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.098965 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:11.100951 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:11.112586 25786 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.113368 25881 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.119486 25712 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:11.120208 25827 ts_tablet_manager.cc:1434] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 12:17:11.101403 26023 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.102902 25896 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.121105 25846 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.121598 25827 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:11.121657 25846 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.103559 26020 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.126402 26020 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.128914 26020 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.130622 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.132133 25786 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:11.145709 25829 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:11.146306 25829 ts_tablet_manager.cc:1403] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.081s	user 0.020s	sys 0.014s
I20260709 12:17:11.149516 25829 raft_consensus.cc:359] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.151271 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.153008 25846 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.155215 25936 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.155850 25936 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:11.156132 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.156877 25936 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:11.157555 25752 leader_election.cc:343] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.150342 25829 raft_consensus.cc:385] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.159318 25829 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.160106 25829 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.164155 25829 ts_tablet_manager.cc:1434] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:17:11.177588 25829 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:11.138393 25894 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.179000 26023 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.181857 26023 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.188100 26023 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.182529 26056 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.188858 26056 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.193708 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.192943 26023 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.194718 26023 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.196260 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:11.196693 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:11.199074 25603 leader_election.cc:343] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:11.199802 25603 leader_election.cc:343] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:11.201231 25677 leader_election.cc:343] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.202075 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.205794 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:11.207330 25603 leader_election.cc:343] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.209235 25786 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:11.213601 25603 leader_election.cc:343] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.205021 25536 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.217516 26056 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.219470 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.220130 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.236802 25638 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:11.224308 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.206856 26023 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.228238 25940 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.230922 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.239179 26020 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.234707 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.240460 25785 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:11.240949 26056 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.242157 25894 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.245839 25829 tablet_bootstrap.cc:654] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:11.253608 25601 leader_election.cc:343] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.254285 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.255318 26059 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:17:11.255841 25601 leader_election.cc:343] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.245227 25828 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:11.243245 25827 tablet_bootstrap.cc:654] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.257043 25828 ts_tablet_manager.cc:1403] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.193s	user 0.017s	sys 0.016s
I20260709 12:17:11.257318 25536 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.251240 25881 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.259178 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:11.259886 25784 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.259179 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.256419 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.263679 26059 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.263782 26017 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.264256 26017 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.264617 26017 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.265861 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.266692 26059 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.267087 26059 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.267449 26059 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.271179 25828 raft_consensus.cc:359] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.271729 25828 raft_consensus.cc:385] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.271947 25828 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.274092 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.278599 25828 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.282459 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:11.283082 25787 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.296092 25846 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.296597 25846 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.298616 25846 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.301482 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:11.303200 25752 leader_election.cc:343] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.303820 25828 ts_tablet_manager.cc:1434] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:17:11.304769 25828 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:11.307297 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.307919 25633 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:11.311998 25751 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.311889 26059 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.312911 25918 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.313653 25918 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.314203 25918 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.316229 25787 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:11.319139 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:11.321552 25603 leader_election.cc:343] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.323274 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.325872 25896 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.325989 25829 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:11.326519 25896 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:11.326776 25829 ts_tablet_manager.cc:1403] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.150s	user 0.012s	sys 0.009s
I20260709 12:17:11.327476 25896 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.329741 25829 raft_consensus.cc:359] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.308028 25784 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:11.330535 25829 raft_consensus.cc:385] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.335976 25829 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.336895 25829 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.342126 25829 ts_tablet_manager.cc:1434] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:17:11.338335 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:11.343537 25829 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:11.343808 26061 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.344460 26061 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:11.348815 26059 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.341539 25918 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.349682 25828 tablet_bootstrap.cc:654] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.358439 26018 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.359724 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:11.350525 26017 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.354857 26061 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.376659 25787 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:11.377913 25827 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:11.378461 25827 ts_tablet_manager.cc:1403] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.257s	user 0.010s	sys 0.011s
I20260709 12:17:11.378181 26015 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.380990 26017 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.382076 25827 raft_consensus.cc:359] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.383292 25827 raft_consensus.cc:385] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.383924 25827 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.384712 25827 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.387413 25827 ts_tablet_manager.cc:1434] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:17:11.385764 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:11.388417 25827 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:11.390935 25829 tablet_bootstrap.cc:654] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.391498 25532 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:11.390568 25752 leader_election.cc:343] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.405164 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:11.405987 25638 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:11.408353 25828 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:11.408878 25828 ts_tablet_manager.cc:1403] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.104s	user 0.018s	sys 0.004s
I20260709 12:17:11.412979 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:11.413633 25636 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.415505 25918 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.419837 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:11.423475 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.424373 25896 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:17:11.425173 25752 leader_election.cc:343] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.428017 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.438210 26017 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.439237 25636 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:11.440616 25751 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.444123 25918 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.444867 25918 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:11.446170 25918 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.447191 25846 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.447718 25846 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.450513 25936 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.451072 25936 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.461498 25846 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.461819 25827 tablet_bootstrap.cc:654] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.462848 25829 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:11.462631 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.466223 25829 ts_tablet_manager.cc:1403] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.123s	user 0.014s	sys 0.026s
I20260709 12:17:11.469113 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.469962 25636 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
W20260709 12:17:11.470824 25752 leader_election.cc:343] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.472642 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:11.474433 25603 leader_election.cc:343] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.475524 25936 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.476318 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.477161 26015 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.477628 26015 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.477591 25828 raft_consensus.cc:359] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.478312 26015 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.478390 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.478596 25828 raft_consensus.cc:385] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.480240 25828 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.481024 25828 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.482636 25828 ts_tablet_manager.cc:1434] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.004s
I20260709 12:17:11.483562 25828 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:11.483842 25533 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:11.487339 25601 leader_election.cc:343] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.491752 25829 raft_consensus.cc:359] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.492396 25829 raft_consensus.cc:385] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.492637 25829 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.493338 25829 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.494597 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.496814 25829 ts_tablet_manager.cc:1434] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:17:11.498368 25829 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:11.499478 25936 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.508679 25828 tablet_bootstrap.cc:654] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.515177 25829 tablet_bootstrap.cc:654] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.515445 25827 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:11.515947 25827 ts_tablet_manager.cc:1403] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.128s	user 0.011s	sys 0.014s
I20260709 12:17:11.526714 25536 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.530031 25827 raft_consensus.cc:359] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.534559 25827 raft_consensus.cc:385] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.534925 25827 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.536015 25827 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.537833 25827 ts_tablet_manager.cc:1434] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:17:11.538739 25827 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:11.540237 25828 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:11.540946 25828 ts_tablet_manager.cc:1403] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.058s	user 0.017s	sys 0.004s
I20260709 12:17:11.549376 25828 raft_consensus.cc:359] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.550158 25828 raft_consensus.cc:385] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.550513 25828 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.551877 25827 tablet_bootstrap.cc:654] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.552299 25828 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.554502 25828 ts_tablet_manager.cc:1434] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:17:11.555992 25828 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Bootstrap starting.
W20260709 12:17:11.557639 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:11.564863 25846 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.520494 26015 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.564831 25828 tablet_bootstrap.cc:654] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.580709 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:11.582703 25752 leader_election.cc:343] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.583523 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:11.583854 26015 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.584383 25633 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.604173 25711 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.604868 25829 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:11.605360 25829 ts_tablet_manager.cc:1403] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.108s	user 0.010s	sys 0.018s
I20260709 12:17:11.607219 25918 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.615710 25827 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:11.616353 25827 ts_tablet_manager.cc:1403] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.078s	user 0.019s	sys 0.037s
I20260709 12:17:11.616068 25936 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.617831 25633 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:11.619602 25827 raft_consensus.cc:359] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.623095 25828 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:11.626679 25827 raft_consensus.cc:385] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.626897 25828 ts_tablet_manager.cc:1403] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.009s
I20260709 12:17:11.627009 25827 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.627720 25827 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.613883 25829 raft_consensus.cc:359] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.628968 25829 raft_consensus.cc:385] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.629251 25829 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.629823 25827 ts_tablet_manager.cc:1434] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.000s
I20260709 12:17:11.630671 25827 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:11.630856 25828 raft_consensus.cc:359] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.631508 25828 raft_consensus.cc:385] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.631783 25828 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.636766 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.641100 25918 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.641736 25918 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:11.642592 25918 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.646006 25827 tablet_bootstrap.cc:654] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.648430 25896 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.657526 25896 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.657992 25896 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.660125 25896 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.662415 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.663414 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:11.663722 25603 leader_election.cc:343] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.663998 25787 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:11.666330 25846 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.666769 25846 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.668562 25846 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.669986 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.671357 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.671986 25633 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:11.672821 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.672818 25896 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.673959 25896 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.674315 25896 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.674692 26061 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.675222 26061 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.676692 25827 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:11.677202 25827 ts_tablet_manager.cc:1403] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.047s	user 0.024s	sys 0.012s
I20260709 12:17:11.679313 26061 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.679780 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.680063 25827 raft_consensus.cc:359] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.680440 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:11.680595 25633 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
W20260709 12:17:11.681269 25752 leader_election.cc:343] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.680851 25827 raft_consensus.cc:385] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.681953 25827 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.681783 26015 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.681751 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:11.682615 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.682785 25827 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.683988 26011 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.684366 26011 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:11.684419 25676 leader_election.cc:343] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.684721 26011 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.682750 26015 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.685734 26015 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.685966 25828 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.688812 25828 ts_tablet_manager.cc:1434] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 12:17:11.682657 25829 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.691134 25829 ts_tablet_manager.cc:1434] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.085s	user 0.010s	sys 0.000s
I20260709 12:17:11.692075 25829 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:11.692842 26015 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.693506 26011 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.695969 25827 ts_tablet_manager.cc:1434] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:17:11.696900 25827 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:11.697228 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:11.697834 25829 tablet_bootstrap.cc:654] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.698757 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:11.704869 25827 tablet_bootstrap.cc:654] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.704730 25634 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.700203 25828 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Bootstrap starting.
W20260709 12:17:11.698676 25752 leader_election.cc:343] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.705938 26015 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.709290 25829 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:11.709774 25829 ts_tablet_manager.cc:1403] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.006s
I20260709 12:17:11.711401 25828 tablet_bootstrap.cc:654] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.712203 25829 raft_consensus.cc:359] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.712844 25829 raft_consensus.cc:385] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.713114 25829 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.713732 25829 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.700462 26011 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.703327 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:11.715946 25633 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.716370 25829 ts_tablet_manager.cc:1434] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:17:11.717275 25829 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:11.703703 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:11.699244 25896 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:11.719990 25676 leader_election.cc:343] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.721935 25896 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.722671 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:11.723273 25829 tablet_bootstrap.cc:654] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.723544 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:11.724119 25787 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:11.724254 25603 leader_election.cc:343] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.713578 25536 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.727079 26061 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.727651 26061 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.729648 26061 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.731099 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.733433 25787 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:11.734853 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.759326 25896 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.759938 25896 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:11.760833 25896 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.735752 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.763729 25636 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:11.744040 25633 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:11.745500 25634 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:11.765522 25677 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:11.757045 25827 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:11.766871 25827 ts_tablet_manager.cc:1403] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.009s
I20260709 12:17:11.767115 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:17:11.768158 26011 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.775861 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.776993 25846 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.777475 25846 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:11.778249 25846 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.778815 26061 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.779587 26061 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:11.780745 26061 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.784543 25710 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.790369 25827 raft_consensus.cc:359] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.791128 25827 raft_consensus.cc:385] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.791401 25827 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.792603 26056 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.793121 26056 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.793354 25535 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.795876 26056 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.796337 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.801245 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.814010 25786 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:11.815842 25827 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.809531 25536 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.808651 26011 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.827580 26056 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.828197 26056 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.829730 26015 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.830571 26015 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.831558 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:11.833192 25603 leader_election.cc:343] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.834197 26056 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.835269 25601 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.836522 25936 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.837221 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.836898 25882 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.838037 25786 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:11.843605 25936 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.843364 26015 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:11.844780 25827 ts_tablet_manager.cc:1434] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.000s
I20260709 12:17:11.849023 25827 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:11.847898 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:11.852800 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.858613 26056 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.861829 26056 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.862205 26056 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:11.864315 25752 leader_election.cc:343] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.861094 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:11.860404 26061 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:11.869691 25751 leader_election.cc:343] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.870167 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:11.872061 26015 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.875211 26061 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.881597 25536 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.890123 25828 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:11.897379 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.899744 26061 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.899995 25829 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:11.970469 25829 ts_tablet_manager.cc:1403] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.253s	user 0.103s	sys 0.083s
I20260709 12:17:11.971215 25828 ts_tablet_manager.cc:1403] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.271s	user 0.037s	sys 0.023s
I20260709 12:17:11.975596 25828 raft_consensus.cc:359] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.975517 26056 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.976365 25828 raft_consensus.cc:385] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.976748 25828 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.977537 25828 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.975858 26059 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.979893 25828 ts_tablet_manager.cc:1434] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:11.947275 25882 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.981731 25828 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:11.980007 26095 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.983351 26095 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.986042 26095 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.985821 26059 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.988644 25827 tablet_bootstrap.cc:654] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:11.972697 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 98673.2us
I20260709 12:17:11.988672 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:11.990608 26056 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:11.991957 25784 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:11.984998 25882 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.998368 25829 raft_consensus.cc:359] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:11.999651 25829 raft_consensus.cc:385] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.999931 25829 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.998354 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:12.001041 25784 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.003080 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:12.004418 25603 leader_election.cc:343] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.005172 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:12.006011 25635 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:12.005934 25829 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.009658 25829 ts_tablet_manager.cc:1434] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.004s
I20260709 12:17:12.014565 25829 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:12.011322 26095 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:17:12.011965 25677 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:12.026400 26095 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.010999 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:12.029938 25784 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:12.031100 25827 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:12.033576 25827 ts_tablet_manager.cc:1403] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.188s	user 0.022s	sys 0.011s
I20260709 12:17:12.057499 25882 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.058295 26017 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.059988 25827 raft_consensus.cc:359] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.060904 25827 raft_consensus.cc:385] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.061250 25827 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.062086 25827 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.065565 25827 ts_tablet_manager.cc:1434] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 12:17:12.065387 26017 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.068581 25829 tablet_bootstrap.cc:654] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.071596 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.072561 26056 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.073051 26056 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:12.073632 25827 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:12.073897 26056 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.081001 25828 tablet_bootstrap.cc:654] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.091939 25634 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:12.093076 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:12.118456 25827 tablet_bootstrap.cc:654] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.141283 26095 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:12.138859 25677 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:12.151443 25882 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.164189 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:12.170410 25677 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:12.171926 26096 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.180541 25709 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:12.206440 25676 consensus_peers.cc:597] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:12.279139 25918 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.280102 25882 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:12.281203 25918 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.281888 25918 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.288988 25829 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:12.289511 25829 ts_tablet_manager.cc:1403] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.275s	user 0.024s	sys 0.028s
I20260709 12:17:12.290570 25918 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:12.292379 25829 raft_consensus.cc:359] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.293121 25829 raft_consensus.cc:385] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.293398 25829 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.294073 25829 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.294848 26096 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.295714 25829 ts_tablet_manager.cc:1434] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:17:12.296680 25829 tablet_bootstrap.cc:492] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:12.299871 25828 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:12.300359 25828 ts_tablet_manager.cc:1403] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.320s	user 0.013s	sys 0.019s
I20260709 12:17:12.308569 25535 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.310401 26095 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.310894 26095 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.315837 26095 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.318019 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:12.318753 25787 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:12.319974 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:12.325074 26095 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.325523 26095 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.325953 26095 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.331900 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:12.341322 25896 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.341452 25828 raft_consensus.cc:359] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.342147 25828 raft_consensus.cc:385] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.342418 25828 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.343246 25827 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:12.344008 25827 ts_tablet_manager.cc:1403] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.271s	user 0.056s	sys 0.055s
I20260709 12:17:12.348536 25829 tablet_bootstrap.cc:654] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.350338 25828 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.360812 25828 ts_tablet_manager.cc:1434] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 12:17:12.362195 25828 tablet_bootstrap.cc:492] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:12.369561 25827 raft_consensus.cc:359] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.370708 25827 raft_consensus.cc:385] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.371048 25827 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.372604 25896 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.374012 25827 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.375773 25896 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.382807 25710 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.384208 25896 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.385728 25827 ts_tablet_manager.cc:1434] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:17:12.386873 25827 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:12.389261 25829 tablet_bootstrap.cc:492] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:12.389710 25829 ts_tablet_manager.cc:1403] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.013s
I20260709 12:17:12.395618 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:12.396353 25635 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:12.404767 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:12.404945 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:12.406208 25918 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.406630 25918 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.377821 26095 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.406996 25918 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.406905 26109 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.408128 25709 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:17:12.409409 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:12.410135 25786 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
W20260709 12:17:12.409451 25603 leader_election.cc:343] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.413156 25829 raft_consensus.cc:359] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.413906 25829 raft_consensus.cc:385] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.414215 25829 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.414968 25829 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.417357 25829 ts_tablet_manager.cc:1434] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.003s
I20260709 12:17:12.419034 26015 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.415741 25896 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:17:12.422917 25829 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:12.423025 25896 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.424396 25896 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:12.454136 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.460142 26015 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:17:12.469283 26095 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.470330 25827 tablet_bootstrap.cc:654] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.460775 26109 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.439910 25896 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.476878 25846 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.466078 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:12.475317 26109 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.481848 26109 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.482756 25896 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.481690 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:12.479048 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:12.490218 25828 tablet_bootstrap.cc:654] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.501189 25918 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.504323 25918 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:12.505443 25638 raft_consensus.cc:2410] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:17:12.521250 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:12.524863 25882 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:12.531118 25752 leader_election.cc:343] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.534602 26015 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.527067 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:12.535279 26015 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.536633 25633 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.538772 26015 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:12.501189 26109 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.546844 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:12.550448 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:12.553579 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:12.554977 25603 leader_election.cc:343] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.559842 26109 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.562817 25829 tablet_bootstrap.cc:654] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.567382 25633 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:12.568444 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:12.569753 25828 tablet_bootstrap.cc:492] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:12.570225 25828 ts_tablet_manager.cc:1403] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.208s	user 0.010s	sys 0.005s
I20260709 12:17:12.572806 25828 raft_consensus.cc:359] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.573253 25827 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:12.573441 25828 raft_consensus.cc:385] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.573673 25827 ts_tablet_manager.cc:1403] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.187s	user 0.019s	sys 0.000s
I20260709 12:17:12.573684 25828 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.575212 25828 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.576094 25827 raft_consensus.cc:359] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.576740 25827 raft_consensus.cc:385] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.576978 25827 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.577638 25829 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:12.577558 25827 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.578049 25829 ts_tablet_manager.cc:1403] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.156s	user 0.008s	sys 0.003s
I20260709 12:17:12.581269 25828 ts_tablet_manager.cc:1434] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:17:12.583102 25828 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:12.585587 25829 raft_consensus.cc:359] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.586347 25829 raft_consensus.cc:385] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.590906 26109 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.591355 26109 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.593230 26109 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.593890 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:12.594640 25786 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.594901 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:12.600353 25603 leader_election.cc:343] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.586683 25829 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.601542 25829 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.603230 25829 ts_tablet_manager.cc:1434] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:17:12.604094 25829 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:12.605382 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:12.606238 25787 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
W20260709 12:17:12.611454 25752 leader_election.cc:343] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.612036 25828 tablet_bootstrap.cc:654] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.613557 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.614439 26109 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.614841 26109 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.615192 26109 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.621789 26109 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.642686 25827 ts_tablet_manager.cc:1434] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 12:17:12.645388 25710 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.646926 25829 tablet_bootstrap.cc:654] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.648609 25827 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:12.649375 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.651031 25896 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.656822 25846 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.657235 25846 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:12.657250 25751 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.658118 25846 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.679926 25828 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:12.680483 25828 ts_tablet_manager.cc:1403] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.014s
I20260709 12:17:12.682629 25828 raft_consensus.cc:359] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.683351 25828 raft_consensus.cc:385] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.683657 25828 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.666522 26115 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:12.690829 25936 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.701140 25882 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.701660 25882 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.703792 25882 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.707000 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:12.709297 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:12.707000 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:17:12.711709 25603 leader_election.cc:343] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.719522 26109 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.720708 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:12.723567 25787 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.727278 25677 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:12.730232 25787 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:12.731434 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.733021 25828 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:12.734541 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:12.739226 26109 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.745234 26109 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:12.746095 26109 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.748391 25827 tablet_bootstrap.cc:654] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.750023 25828 ts_tablet_manager.cc:1434] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.000s
I20260709 12:17:12.716777 25786 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:12.751842 25828 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:12.752483 26093 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.766156 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.768278 25936 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.768947 25936 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:12.770071 25829 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:12.770583 25829 ts_tablet_manager.cc:1403] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.167s	user 0.009s	sys 0.011s
I20260709 12:17:12.766126 25784 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.771309 25936 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.773437 25829 raft_consensus.cc:359] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.774063 25829 raft_consensus.cc:385] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.774356 25829 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.770610 25536 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:12.796101 25535 catalog_manager.cc:5697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.803359 25829 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.831182 25829 ts_tablet_manager.cc:1434] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 12:17:12.832818 25829 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:12.832937 25828 tablet_bootstrap.cc:654] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.838937 25936 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.839757 25827 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:12.839442 25936 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.840276 25827 ts_tablet_manager.cc:1403] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.197s	user 0.044s	sys 0.034s
I20260709 12:17:12.841544 25936 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.841905 25829 tablet_bootstrap.cc:654] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.843010 25827 raft_consensus.cc:359] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.843739 25827 raft_consensus.cc:385] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.843992 25827 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.844599 25827 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.846850 25827 ts_tablet_manager.cc:1434] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:12.847746 25827 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:12.848071 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:12.849709 25603 leader_election.cc:343] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.852597 25828 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:12.854110 25828 ts_tablet_manager.cc:1403] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.005s
I20260709 12:17:12.856932 25828 raft_consensus.cc:359] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.857594 25828 raft_consensus.cc:385] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.857857 25828 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.858528 25828 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.859387 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:12.860284 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:12.860491 25828 ts_tablet_manager.cc:1434] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:17:12.861474 25828 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:12.861800 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.862825 25936 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.863181 25936 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.863502 25936 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.869351 25827 tablet_bootstrap.cc:654] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.871726 25936 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.874171 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:12.875406 25603 leader_election.cc:343] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.878520 25936 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.883921 25828 tablet_bootstrap.cc:654] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.881151 25535 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.884951 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:12.885571 25784 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.894018 25829 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:12.894152 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:12.894541 25829 ts_tablet_manager.cc:1403] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.012s
I20260709 12:17:12.895313 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.896266 25936 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.896648 25936 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:12.897184 25829 raft_consensus.cc:359] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.897363 25936 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.897894 25829 raft_consensus.cc:385] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.898154 25829 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.898818 25829 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.909307 25829 ts_tablet_manager.cc:1434] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 12:17:12.910207 25829 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:12.941606 25829 tablet_bootstrap.cc:654] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.943459 26109 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.943974 26109 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:12.946004 26109 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:12.952109 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:12.954715 25603 leader_election.cc:343] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.962347 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:12.963238 25787 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:12.983018 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:12.984138 25936 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.984632 25936 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.985250 25936 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.999344 25535 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.007680 25936 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.019752 25936 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.022054 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:13.022737 25787 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.027451 25787 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:13.029130 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:13.030870 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:13.032639 25936 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.033279 25936 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:13.034200 25936 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.048067 25828 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:13.048712 25828 ts_tablet_manager.cc:1403] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.187s	user 0.036s	sys 0.035s
I20260709 12:17:13.053850 25828 raft_consensus.cc:359] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.054625 25828 raft_consensus.cc:385] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.061496 25827 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:13.062074 25827 ts_tablet_manager.cc:1403] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.215s	user 0.035s	sys 0.074s
I20260709 12:17:13.054940 25828 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
W20260709 12:17:13.064699 25603 leader_election.cc:343] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:13.064811 25752 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:13.065255 25828 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.069511 25829 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:13.070052 25829 ts_tablet_manager.cc:1403] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.160s	user 0.026s	sys 0.013s
I20260709 12:17:13.073668 25828 ts_tablet_manager.cc:1434] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:17:13.074635 25828 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:13.075354 25829 raft_consensus.cc:359] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.076006 25829 raft_consensus.cc:385] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.076609 25829 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.077322 25829 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.081328 25829 ts_tablet_manager.cc:1434] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:17:13.081843 25828 tablet_bootstrap.cc:654] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.082257 25829 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:13.075583 25827 raft_consensus.cc:359] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.084471 25827 raft_consensus.cc:385] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.084779 25827 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.085472 25827 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.087685 25827 ts_tablet_manager.cc:1434] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:17:13.088071 25829 tablet_bootstrap.cc:654] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.088833 25827 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:13.092334 25828 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:13.092957 25828 ts_tablet_manager.cc:1403] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.003s
I20260709 12:17:13.150683 25827 tablet_bootstrap.cc:654] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.168061 26093 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.169790 26093 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.181097 25936 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.181330 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:13.182327 25936 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.186779 26093 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.191767 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.193070 25786 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:13.193977 26056 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:17:13.194602 25676 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:13.195711 26093 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.200469 25536 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.205971 26056 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.211572 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:13.214327 25603 leader_election.cc:343] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.208055 25936 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.209592 26056 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.228556 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.227891 25828 raft_consensus.cc:359] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.236274 25828 raft_consensus.cc:385] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.236712 25828 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.243343 25829 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
W20260709 12:17:13.246167 25603 leader_election.cc:343] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.245327 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.252986 26093 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.248632 25829 ts_tablet_manager.cc:1403] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.167s	user 0.013s	sys 0.010s
I20260709 12:17:13.259100 26056 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.259562 25828 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.261418 25786 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
W20260709 12:17:13.277606 25603 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:13.284386 25828 ts_tablet_manager.cc:1434] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.191s	user 0.008s	sys 0.002s
I20260709 12:17:13.286142 25828 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:13.287849 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.288868 25786 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:13.290942 25827 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:13.292806 26015 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.298509 25828 tablet_bootstrap.cc:654] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.317967 26056 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.318050 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:13.316792 25829 raft_consensus.cc:359] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.320214 25829 raft_consensus.cc:385] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.320453 25936 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.320588 25829 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.320956 25936 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.321496 25829 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.323251 25827 ts_tablet_manager.cc:1403] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.235s	user 0.025s	sys 0.002s
I20260709 12:17:13.323560 25936 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.324266 25829 ts_tablet_manager.cc:1434] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 12:17:13.325423 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.326023 25829 tablet_bootstrap.cc:492] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
W20260709 12:17:13.327131 25603 leader_election.cc:343] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.327080 25827 raft_consensus.cc:359] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.328140 26109 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.328547 26109 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.328971 26109 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.329500 25827 raft_consensus.cc:385] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.329883 25827 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.332952 25827 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.334285 25601 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:13.335093 26056 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.335453 26056 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.335821 26056 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.344137 26056 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.347065 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:13.348322 25829 tablet_bootstrap.cc:654] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:13.348501 25603 leader_election.cc:343] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.348783 25828 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:13.349309 25828 ts_tablet_manager.cc:1403] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.006s
I20260709 12:17:13.347084 25827 ts_tablet_manager.cc:1434] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.003s
I20260709 12:17:13.353268 25828 raft_consensus.cc:359] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.354022 25828 raft_consensus.cc:385] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.354352 25828 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.358621 26056 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.357028 25829 tablet_bootstrap.cc:492] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:13.351836 26109 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.360975 25829 ts_tablet_manager.cc:1403] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.035s	user 0.001s	sys 0.014s
I20260709 12:17:13.355521 25828 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.365513 25828 ts_tablet_manager.cc:1434] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:17:13.366849 25828 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:13.363369 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:13.372351 25827 tablet_bootstrap.cc:492] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:13.373068 25828 tablet_bootstrap.cc:654] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:13.372392 25603 leader_election.cc:343] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.381244 25827 tablet_bootstrap.cc:654] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.395110 25827 tablet_bootstrap.cc:492] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:13.395550 25827 ts_tablet_manager.cc:1403] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.003s
I20260709 12:17:13.397112 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:13.397595 26056 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.398202 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:13.398872 25783 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.399268 25827 raft_consensus.cc:359] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.399925 25827 raft_consensus.cc:385] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.400202 25827 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.400348 25829 raft_consensus.cc:359] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.401160 25829 raft_consensus.cc:385] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.400985 25827 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.401508 25829 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.363323 26109 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.395494 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.404086 25787 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:13.408993 26109 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.397701 25785 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.409519 26109 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.412155 26109 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.421355 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.424111 25827 ts_tablet_manager.cc:1434] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:17:13.420420 25829 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.422165 25711 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:13.426748 26015 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.431461 26015 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.431860 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:13.439776 25829 ts_tablet_manager.cc:1434] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.003s
I20260709 12:17:13.444037 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:13.445170 25936 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.445662 25936 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.446045 25936 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.446921 25896 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:13.447521 25896 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.447952 25896 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.454646 26056 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:17:13.476105 25829 tablet_bootstrap.cc:492] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:13.504323 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.507076 25787 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:13.512674 25827 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:13.520449 25828 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:13.520577 26015 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:13.521203 25828 ts_tablet_manager.cc:1403] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.155s	user 0.034s	sys 0.059s
I20260709 12:17:13.526803 25783 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:13.529222 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.529066 25896 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.534680 25896 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:17:13.518313 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 65510.3us
I20260709 12:17:13.537976 25710 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.538201 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:13.540236 25936 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.543037 25711 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:13.544744 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:13.546924 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:13.547870 25827 tablet_bootstrap.cc:654] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.553530 25882 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.554041 25882 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.553828 25785 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:13.556689 25936 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.556895 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:13.557423 26100 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:13.557801 26100 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.558157 26100 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:13.563779 25603 leader_election.cc:343] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.558476 25882 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.568162 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:13.566218 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.575336 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:13.576018 25882 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.576442 25882 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.578038 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:13.578790 25638 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:13.579541 26096 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.580318 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:13.582791 25882 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.583612 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:13.584374 25603 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:13.585247 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:13.589128 26096 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.591501 26100 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.592681 25787 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 2: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:13.598011 25828 raft_consensus.cc:359] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.598775 25828 raft_consensus.cc:385] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.599066 25828 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.602272 25828 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.604807 25829 tablet_bootstrap.cc:654] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.607236 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:13.610304 25601 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:13.612325 25936 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.612699 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:13.615835 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:13.618511 25787 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.612762 25936 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:13.614099 26109 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.631645 25828 ts_tablet_manager.cc:1434] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.110s	user 0.008s	sys 0.000s
I20260709 12:17:13.636884 26156 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.635280 26109 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:13.640033 26100 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:13.640784 26109 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.642951 26093 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.643527 26093 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.645435 26156 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.649528 25787 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:13.650954 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:13.651618 25711 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.652127 25712 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.652571 25936 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.674921 26093 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.676860 25827 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:13.677490 25827 ts_tablet_manager.cc:1403] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.229s	user 0.013s	sys 0.011s
I20260709 12:17:13.682928 25828 tablet_bootstrap.cc:492] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:13.677986 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:13.698086 25827 raft_consensus.cc:359] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.699291 25827 raft_consensus.cc:385] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.699677 25827 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.700861 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.702003 25785 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:13.703527 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:13.707787 26096 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:13.708144 26096 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:13.708508 26096 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.708899 25829 tablet_bootstrap.cc:492] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:13.709349 25829 ts_tablet_manager.cc:1403] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.247s	user 0.011s	sys 0.010s
I20260709 12:17:13.718863 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:13.691428 25711 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:13.721930 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:13.725322 25827 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.726703 25710 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.727933 25829 raft_consensus.cc:359] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.728839 25829 raft_consensus.cc:385] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.730675 25829 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.731515 25829 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.733732 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:13.735150 25635 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:13.736738 26015 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:13.737668 25896 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.738127 25896 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:13.738832 25896 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.744504 25829 ts_tablet_manager.cc:1434] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.003s
I20260709 12:17:13.738667 26015 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:13.749396 25882 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.735792 26056 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.749840 25882 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.749207 26015 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.782866 25827 ts_tablet_manager.cc:1434] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.007s
I20260709 12:17:13.790875 25828 tablet_bootstrap.cc:654] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.795992 25882 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.797175 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.770385 26056 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.809214 25827 tablet_bootstrap.cc:492] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:13.782855 26109 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:17:13.832617 26169 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.833254 26169 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.831678 26156 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.861254 25829 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:13.852833 26169 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.870767 26171 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.871426 26171 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.873660 26056 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.855691 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:13.875331 26171 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:13.871019 25827 tablet_bootstrap.cc:654] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:13.884167 26164 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:17:13.889097 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:17:13.900514 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 142406us
I20260709 12:17:13.907719 26160 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.909037 26161 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.909672 26161 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.912262 25535 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.916827 26096 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.917395 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.921882 26161 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:13.860741 26156 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:13.938916 25752 leader_election.cc:343] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.938696 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.950522 25712 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.940948 25535 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.947906 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.959676 26100 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:13.960064 25828 tablet_bootstrap.cc:492] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:13.960692 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.963214 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.968597 25828 ts_tablet_manager.cc:1403] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.315s	user 0.006s	sys 0.016s
I20260709 12:17:13.979028 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:13.980444 26100 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.980928 25828 raft_consensus.cc:359] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:13.981775 25828 raft_consensus.cc:385] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.982061 25828 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.985857 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:14.000094 25601 leader_election.cc:343] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.989938 25828 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.002677 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:14.003616 25633 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:14.004722 25896 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:17:13.999061 25603 leader_election.cc:343] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.982630 26100 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:17:14.008299 25603 leader_election.cc:343] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.982697 26156 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.008621 25603 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:14.012904 26169 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.994244 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:14.015426 25896 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.987891 25784 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:14.015457 26169 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:13.988516 25711 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:14.027189 26096 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.021409 25634 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:14.021299 26169 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.022292 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:14.022234 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:13.996642 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:14.026788 25829 tablet_bootstrap.cc:654] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.032177 25603 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:14.032966 25785 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:14.033993 25783 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:14.035759 26169 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.050094 25828 ts_tablet_manager.cc:1434] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.003s
I20260709 12:17:14.053922 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:14.057912 25896 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.058295 25896 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.058645 25896 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.060599 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:14.060704 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:14.056488 26109 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:14.063953 26109 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.064364 26109 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.063084 26056 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.067754 26056 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.068116 26056 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:14.081651 25603 leader_election.cc:343] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.115033 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:14.116338 25828 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Bootstrap starting.
W20260709 12:17:14.117182 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:14.121421 26056 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.128089 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:17:14.129201 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 79258.6us
I20260709 12:17:14.129587 25779 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.133021 26056 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.133162 26114 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:14.133759 26114 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:14.134127 26114 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.135839 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:14.136672 26100 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.137053 26100 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.137363 26100 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.140854 25828 tablet_bootstrap.cc:654] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.147951 25827 tablet_bootstrap.cc:492] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.148423 25827 ts_tablet_manager.cc:1403] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.340s	user 0.017s	sys 0.010s
I20260709 12:17:14.156651 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:14.163153 25896 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.167697 25896 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.168738 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:14.171617 26100 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.172845 25785 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:14.174808 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:14.174886 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:14.175734 25634 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:14.176285 25633 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
W20260709 12:17:14.176461 25752 leader_election.cc:343] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.178305 25677 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:17:14.178372 25827 raft_consensus.cc:359] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.179085 25827 raft_consensus.cc:385] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.179169 26093 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:14.179375 25827 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.179672 26093 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:14.181099 26093 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.182606 25707 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.187165 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:17:14.226243 26176 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:17:14.231436 25827 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.223912 26161 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:17:14.235030 25603 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:14.234903 25827 ts_tablet_manager.cc:1434] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.086s	user 0.010s	sys 0.001s
I20260709 12:17:14.240767 25829 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:14.182035 26100 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:14.241408 25829 ts_tablet_manager.cc:1403] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.381s	user 0.027s	sys 0.023s
I20260709 12:17:14.191896 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:14.242437 25827 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.242132 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:14.179571 26109 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.250093 25829 raft_consensus.cc:359] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.250860 25829 raft_consensus.cc:385] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.221339 26114 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.251149 25829 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.246155 25708 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.245836 25779 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:14.255719 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:14.256033 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:14.256599 25710 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:14.258736 26156 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.259940 25829 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.259174 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:14.259213 26156 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:14.263080 25635 raft_consensus.cc:2368] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3.
I20260709 12:17:14.263556 26156 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.265225 26109 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.266523 26146 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 12:17:14.267449 25603 leader_election.cc:343] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.267963 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:14.269945 26056 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.269851 26109 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:14.271703 25828 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:14.272266 25828 ts_tablet_manager.cc:1403] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.198s	user 0.014s	sys 0.002s
I20260709 12:17:14.272583 25751 leader_election.cc:400] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3.
I20260709 12:17:14.272907 25751 leader_election.cc:403] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:17:14.273489 25603 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:14.258172 26114 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:14.280678 26168 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.281675 25829 ts_tablet_manager.cc:1434] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 12:17:14.282950 25827 tablet_bootstrap.cc:654] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.271588 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:14.295156 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:14.296026 25710 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.297590 25635 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:14.299883 26161 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.309304 25710 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:17:14.311079 25828 raft_consensus.cc:359] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.312114 25828 raft_consensus.cc:385] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.312484 25828 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.315202 25828 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.321003 25603 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:17:14.328516 25603 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:14.327661 26156 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:14.332211 25829 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:14.325923 26109 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:14.323356 26168 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3.
I20260709 12:17:14.344008 25710 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:14.345700 26168 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:14.343104 26109 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:14.355489 25828 ts_tablet_manager.cc:1434] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.002s
I20260709 12:17:14.356974 26109 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.379762 25827 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.380324 25827 ts_tablet_manager.cc:1403] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.138s	user 0.018s	sys 0.011s
I20260709 12:17:14.381625 25828 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:14.401120 25536 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 2, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:14.413180 25827 raft_consensus.cc:359] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.414036 25827 raft_consensus.cc:385] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.414316 25827 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.415112 25827 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.417426 25827 ts_tablet_manager.cc:1434] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.002s
I20260709 12:17:14.422874 25827 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.423213 26093 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.424615 26093 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.436308 25829 tablet_bootstrap.cc:654] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.428717 26093 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.437614 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:14.452895 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:14.453661 25786 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:14.454139 25708 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
W20260709 12:17:14.456671 25677 leader_election.cc:343] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.460943 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:14.462201 26093 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.462705 26093 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.463042 26093 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.461040 25781 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:17:14.523772 25828 tablet_bootstrap.cc:654] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.543759 25827 tablet_bootstrap.cc:654] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.570590 25829 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:14.571130 25829 ts_tablet_manager.cc:1403] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.248s	user 0.016s	sys 0.012s
I20260709 12:17:14.555900 26093 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.578450 25829 raft_consensus.cc:359] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.580129 25829 raft_consensus.cc:385] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.580536 25829 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.582834 25829 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.591197 25829 ts_tablet_manager.cc:1434] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.007s
I20260709 12:17:14.595454 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:14.596133 25784 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.594250 26093 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.598305 26109 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.598908 26109 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.602751 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:14.603410 25635 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.605115 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:14.609274 25533 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:14.610984 25603 leader_election.cc:343] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.610952 25533 catalog_manager.cc:5697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.625610 25829 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:14.629791 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:14.630777 25828 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:14.631327 25828 ts_tablet_manager.cc:1403] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.266s	user 0.014s	sys 0.008s
I20260709 12:17:14.612473 25533 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:14.631784 25601 leader_election.cc:343] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.632210 25601 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:14.632231 26109 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.632932 26155 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.637413 25829 tablet_bootstrap.cc:654] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.638517 25828 raft_consensus.cc:359] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.639410 25827 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.639189 25828 raft_consensus.cc:385] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.639909 25827 ts_tablet_manager.cc:1403] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.222s	user 0.012s	sys 0.010s
I20260709 12:17:14.640074 25828 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.639184 25784 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:14.641773 25828 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.642585 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:14.643306 26093 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.643983 26093 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:14.644505 25635 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:14.646981 25828 ts_tablet_manager.cc:1434] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:17:14.646697 26093 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.648361 25828 tablet_bootstrap.cc:492] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:14.653441 25827 raft_consensus.cc:359] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.653944 25827 raft_consensus.cc:385] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.654182 25827 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.655076 25827 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.657240 25827 ts_tablet_manager.cc:1434] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:17:14.658383 25828 tablet_bootstrap.cc:654] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.661499 25827 tablet_bootstrap.cc:492] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.666896 25535 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.670239 25784 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.672170 25828 tablet_bootstrap.cc:492] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:14.672672 25828 ts_tablet_manager.cc:1403] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.000s
I20260709 12:17:14.675530 25828 raft_consensus.cc:359] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.679822 25827 tablet_bootstrap.cc:654] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.676419 25828 raft_consensus.cc:385] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.680696 25828 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.682665 25828 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.684845 26093 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.685702 25828 ts_tablet_manager.cc:1434] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:17:14.685488 26093 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.687857 26093 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.688423 25828 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:14.689822 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:14.696045 25828 tablet_bootstrap.cc:654] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.696461 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:14.698145 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:14.699254 26093 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.702162 25784 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:14.704298 26093 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:14.705399 26109 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:14.717164 25635 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:14.718628 25882 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:14.731298 26169 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.791615 26179 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.792264 26179 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.798749 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:14.799734 25633 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:14.803301 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:14.803558 25535 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.804801 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:17:14.805441 25752 leader_election.cc:343] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.806094 26179 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:14.806635 26168 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.807304 26168 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.807621 26168 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.812885 25827 tablet_bootstrap.cc:492] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.813443 25827 ts_tablet_manager.cc:1403] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.152s	user 0.022s	sys 0.032s
I20260709 12:17:14.815554 25829 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:14.816099 25829 ts_tablet_manager.cc:1403] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.192s	user 0.047s	sys 0.077s
I20260709 12:17:14.816257 26168 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.818886 25829 raft_consensus.cc:359] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.824370 25829 raft_consensus.cc:385] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.824646 25829 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.825371 25829 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.826289 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:14.826964 25829 ts_tablet_manager.cc:1434] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:17:14.827071 25633 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.820077 25827 raft_consensus.cc:359] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.827857 25827 raft_consensus.cc:385] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.828169 25829 tablet_bootstrap.cc:492] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:14.828198 25827 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.829232 25827 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.831990 25827 ts_tablet_manager.cc:1434] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:17:14.833003 25827 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.837082 25829 tablet_bootstrap.cc:654] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.839334 25827 tablet_bootstrap.cc:654] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.847905 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:14.848539 25710 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.844871 26168 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:14.850831 25827 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.851336 25827 ts_tablet_manager.cc:1403] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:17:14.853214 25828 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:14.853781 25828 ts_tablet_manager.cc:1403] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.166s	user 0.037s	sys 0.037s
I20260709 12:17:14.854907 25827 raft_consensus.cc:359] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.855607 25710 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:14.855638 25827 raft_consensus.cc:385] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.856231 25827 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.856890 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:14.856949 25827 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.859434 25827 ts_tablet_manager.cc:1434] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:17:14.860414 25827 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.860344 26168 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.864178 26168 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:14.865419 25829 tablet_bootstrap.cc:492] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:14.866827 25829 ts_tablet_manager.cc:1403] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.009s
I20260709 12:17:14.861824 25633 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:14.867228 25827 tablet_bootstrap.cc:654] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.868635 26168 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.865756 25828 raft_consensus.cc:359] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.870817 25828 raft_consensus.cc:385] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.871124 25828 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.869611 25829 raft_consensus.cc:359] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.871798 25829 raft_consensus.cc:385] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.872102 25829 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.872845 25829 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.874881 25829 ts_tablet_manager.cc:1434] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:17:14.875816 25829 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:14.878865 25828 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.892006 25828 ts_tablet_manager.cc:1434] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.038s	user 0.001s	sys 0.005s
I20260709 12:17:14.894199 25827 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.894744 25827 ts_tablet_manager.cc:1403] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.000s
I20260709 12:17:14.899677 25828 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:14.905931 25827 raft_consensus.cc:359] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.906644 25827 raft_consensus.cc:385] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.907131 25827 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.907753 25827 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.917798 25829 tablet_bootstrap.cc:654] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.921437 25827 ts_tablet_manager.cc:1434] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:17:14.922533 25827 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.919840 25533 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:14.925817 26160 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.926316 26160 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.929733 25827 tablet_bootstrap.cc:654] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.930902 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:14.931751 25634 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:14.934110 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:14.934868 25786 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:14.935930 26160 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:14.935993 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:14.937361 26093 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.942131 25828 tablet_bootstrap.cc:654] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.943454 25708 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:14.945024 25827 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.945487 25827 ts_tablet_manager.cc:1403] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.006s
I20260709 12:17:14.948469 25827 raft_consensus.cc:359] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.949332 25827 raft_consensus.cc:385] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.950476 26146 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.952219 26093 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.949680 25827 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.953955 25827 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.956142 25827 ts_tablet_manager.cc:1434] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:17:14.957131 25827 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.959913 25828 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:14.960544 25828 ts_tablet_manager.cc:1403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.005s
I20260709 12:17:14.962905 25827 tablet_bootstrap.cc:654] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.964265 25828 raft_consensus.cc:359] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.964789 25829 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:14.964989 25828 raft_consensus.cc:385] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.965262 25828 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.965947 25828 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.969528 25827 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:14.970093 25827 ts_tablet_manager.cc:1403] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 12:17:14.971611 25786 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:14.973246 26169 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:14.975528 25827 raft_consensus.cc:359] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.976233 25827 raft_consensus.cc:385] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.976586 25827 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.978952 25828 ts_tablet_manager.cc:1434] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:17:14.980060 25828 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:14.983801 25827 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:14.986668 25827 ts_tablet_manager.cc:1434] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:17:14.988137 25827 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:14.988483 25828 tablet_bootstrap.cc:654] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.965270 25829 ts_tablet_manager.cc:1403] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.090s	user 0.001s	sys 0.011s
I20260709 12:17:15.000849 26161 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.001482 26161 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.003870 26161 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.006933 25827 tablet_bootstrap.cc:654] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.006727 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.011853 25633 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:15.013093 25751 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:15.013962 26179 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.014390 26179 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.014801 26179 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.017450 25828 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:15.020516 25828 ts_tablet_manager.cc:1403] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.041s	user 0.028s	sys 0.005s
I20260709 12:17:15.023334 25828 raft_consensus.cc:359] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.024037 25828 raft_consensus.cc:385] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.025493 25828 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.026268 25828 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.028852 25828 ts_tablet_manager.cc:1434] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:15.030148 25828 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:15.005663 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:15.034054 26093 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.034646 26093 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.037408 26093 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:17:15.038106 25752 leader_election.cc:343] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.000144 25829 raft_consensus.cc:359] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.040117 25829 raft_consensus.cc:385] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.040402 25829 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.041038 25829 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.042085 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.043133 26169 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.043807 25829 ts_tablet_manager.cc:1434] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.002s
I20260709 12:17:15.043736 26169 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.045254 25829 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:15.046504 25828 tablet_bootstrap.cc:654] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.046217 26169 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.046949 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:15.048460 25708 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:15.048494 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:15.049943 25603 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.050212 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:15.050877 26169 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.051251 26093 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.051678 26169 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.052333 26169 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.053326 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:15.053942 25784 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:15.064453 26109 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.065193 25827 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:15.065127 26109 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.065882 25827 ts_tablet_manager.cc:1403] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.078s	user 0.022s	sys 0.008s
I20260709 12:17:15.068656 25827 raft_consensus.cc:359] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.069325 25827 raft_consensus.cc:385] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.069612 25827 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.069708 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:15.070261 25827 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:15.071321 25603 leader_election.cc:343] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.072422 25827 ts_tablet_manager.cc:1434] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:17:15.071834 26179 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.073817 26109 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.075851 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:15.076314 25882 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.076745 25882 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.079319 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.087823 25634 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:15.087706 26169 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.089246 25677 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.089529 26179 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.090114 26093 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.083910 25829 tablet_bootstrap.cc:654] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.090786 26093 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.091150 26093 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.091265 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:15.092927 25827 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:15.096735 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:15.097409 25707 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.086972 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:15.078464 25882 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.099311 25784 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:15.104135 25706 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.104681 26169 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.099305 25827 tablet_bootstrap.cc:654] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:15.077379 25601 leader_election.cc:343] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.106153 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:15.108175 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:15.108822 25784 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.115018 25828 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:15.115648 25828 ts_tablet_manager.cc:1403] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.086s	user 0.034s	sys 0.020s
I20260709 12:17:15.116254 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:15.116906 25634 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.118927 25828 raft_consensus.cc:359] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.119890 25828 raft_consensus.cc:385] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.120291 25828 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.121325 25828 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.129429 25707 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:15.130772 25603 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.131592 26109 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.131781 25828 ts_tablet_manager.cc:1434] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:17:15.132274 26109 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.132745 26109 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:15.133262 25828 tablet_bootstrap.cc:492] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:15.133107 26160 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.133632 26109 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.162774 25828 tablet_bootstrap.cc:654] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.168485 25827 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:15.169162 25827 ts_tablet_manager.cc:1403] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.076s	user 0.013s	sys 0.018s
I20260709 12:17:15.172406 25829 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:15.172981 25829 ts_tablet_manager.cc:1403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.128s	user 0.012s	sys 0.022s
I20260709 12:17:15.175128 25828 tablet_bootstrap.cc:492] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:15.175585 25828 ts_tablet_manager.cc:1403] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.003s
I20260709 12:17:15.175726 25829 raft_consensus.cc:359] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.176477 25829 raft_consensus.cc:385] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.176767 25829 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.177485 25829 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.178239 25828 raft_consensus.cc:359] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.179013 25828 raft_consensus.cc:385] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.179289 25828 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.187486 25828 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.188215 25829 ts_tablet_manager.cc:1434] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 12:17:15.189154 25828 ts_tablet_manager.cc:1434] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.003s
I20260709 12:17:15.189309 25827 raft_consensus.cc:359] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.189952 25827 raft_consensus.cc:385] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.190166 25828 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:15.190229 25827 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.191066 25827 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.098246 26093 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.193017 25827 ts_tablet_manager.cc:1434] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.003s
I20260709 12:17:15.194280 25827 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:15.199005 25710 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:15.199057 25829 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:15.200160 25828 tablet_bootstrap.cc:654] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.204766 26160 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.205605 26160 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.206415 25827 tablet_bootstrap.cc:654] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.223676 25708 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.232100 25828 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:15.241231 25828 ts_tablet_manager.cc:1403] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.010s
I20260709 12:17:15.242745 25829 tablet_bootstrap.cc:654] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.258689 25784 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:15.251101 26161 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.275558 26161 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.277897 26161 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.277362 25532 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.293715 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:15.294626 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.295701 25784 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:15.297350 25633 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:15.287799 26179 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.295290 26160 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.312306 25827 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:15.313022 25827 ts_tablet_manager.cc:1403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.003s
I20260709 12:17:15.295209 26015 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.320719 26093 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.323536 26156 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.324218 25634 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:15.328063 26015 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.338929 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.339977 25636 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:15.332697 26156 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.344197 26156 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.347383 26015 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.352839 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:15.355311 25706 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:15.356905 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:15.354940 25827 raft_consensus.cc:359] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.362793 25827 raft_consensus.cc:385] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.363180 25827 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.363945 25827 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.317107 26179 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.365010 25710 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:15.366578 25827 ts_tablet_manager.cc:1434] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.053s	user 0.010s	sys 0.001s
I20260709 12:17:15.367331 26179 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.367887 25827 tablet_bootstrap.cc:492] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:15.368160 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.369040 26155 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.369437 26155 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.369753 26155 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.379019 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.379935 25882 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.380399 25882 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.380759 25882 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.382993 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:15.383677 25784 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
W20260709 12:17:15.385821 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 89262.2us
I20260709 12:17:15.389202 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:15.390225 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.390793 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.404731 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:15.407642 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:15.407663 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:15.411523 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:15.412206 25707 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
W20260709 12:17:15.414475 25752 leader_election.cc:343] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.420521 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.422430 25751 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:15.425050 25828 raft_consensus.cc:359] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.426079 25828 raft_consensus.cc:385] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.426460 25828 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.420760 25636 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:15.431140 25751 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:15.436435 25882 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.438076 25710 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:15.427642 25783 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:15.427314 26168 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.413498 26230 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:17:15.458709 26232 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.459286 26232 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:15.460100 26232 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.451452 25828 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.465942 25827 tablet_bootstrap.cc:654] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.439038 26231 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.468583 25882 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.451007 26168 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.481456 25828 ts_tablet_manager.cc:1434] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.240s	user 0.007s	sys 0.003s
I20260709 12:17:15.468312 26231 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.512567 25828 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Bootstrap starting.
I20260709 12:17:15.480520 26168 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:15.579452 26233 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:15.521729 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:15.518756 26231 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.607501 25636 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.634409 25828 tablet_bootstrap.cc:654] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.614766 26015 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:17:15.628885 26155 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.693607 25638 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:15.706405 26155 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.715862 25828 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: No bootstrap required, opened a new log
I20260709 12:17:15.716640 25828 ts_tablet_manager.cc:1403] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Time spent bootstrapping tablet: real 0.225s	user 0.021s	sys 0.009s
I20260709 12:17:15.706849 26232 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.709642 26096 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:15.731681 25636 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:15.735483 25828 raft_consensus.cc:359] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.738198 25828 raft_consensus.cc:385] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.738668 25828 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.739589 25828 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.728183 26096 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.751524 25828 ts_tablet_manager.cc:1434] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.002s
I20260709 12:17:15.727406 26232 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.776448 26232 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.777873 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:15.779615 25752 leader_election.cc:343] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.779062 26168 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.782915 26168 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.784411 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:15.785516 26096 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:15.785328 25711 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.774976 26231 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.795514 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:15.800138 25711 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:15.796778 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:15.801836 25783 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:15.805385 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.808207 26168 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.808663 26168 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:15.809228 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:15.809433 26168 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.811636 25882 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.811805 26241 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:15.813436 26231 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:15.815344 25882 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:15.816171 25882 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.817186 25829 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:15.817647 25829 ts_tablet_manager.cc:1403] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.619s	user 0.008s	sys 0.011s
I20260709 12:17:15.819453 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:15.820612 25829 raft_consensus.cc:359] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.821499 25829 raft_consensus.cc:385] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.821782 25829 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.822538 25829 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.825819 25829 ts_tablet_manager.cc:1434] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:17:15.826948 25829 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:15.827174 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.831316 26169 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.831815 26169 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:15.834746 26169 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:17:15.835186 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 404207us
W20260709 12:17:15.836525 25751 leader_election.cc:343] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.836915 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:15.851890 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:15.852768 25784 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:15.854700 26231 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:17:15.860116 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 69542.8us
I20260709 12:17:15.864161 25827 tablet_bootstrap.cc:492] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:15.864609 25827 ts_tablet_manager.cc:1403] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.497s	user 0.009s	sys 0.027s
I20260709 12:17:15.866259 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:15.878075 26093 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.875447 25532 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:15.892889 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:15.915406 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:15.916620 25783 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:15.907719 26169 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.922380 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:15.924005 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:15.930577 25631 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:15.934012 25633 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:15.937484 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:15.938423 25629 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
I20260709 12:17:15.997762 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:15.998994 26109 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:15.999433 26109 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.999811 26109 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.005883 26109 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.008088 25827 raft_consensus.cc:359] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.010134 25827 raft_consensus.cc:385] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.010545 25827 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.010972 26239 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.011375 25827 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.036604 25827 ts_tablet_manager.cc:1434] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.172s	user 0.005s	sys 0.016s
I20260709 12:17:16.054255 25829 tablet_bootstrap.cc:654] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.011565 26239 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.055896 25827 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:16.057301 26239 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:16.058708 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:16.065918 25706 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.061582 25751 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:16.073057 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:16.074919 26239 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.074378 26161 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.075542 26239 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.076064 26239 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.078164 26015 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:17:16.102771 25706 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
W20260709 12:17:16.162612 26256 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.098s	user 0.002s	sys 0.011s
W20260709 12:17:16.173333 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 100349us
I20260709 12:17:16.192366 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:16.199751 26253 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.201435 26015 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:16.213652 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:16.215734 25752 leader_election.cc:343] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.215844 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:16.216782 26109 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:16.222859 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:16.223495 25783 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.233752 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:16.240101 26237 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.238250 25676 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:16.240726 26237 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.242272 26096 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.262442 25533 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.268170 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:16.268949 26146 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:16.269407 26146 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:16.270213 26146 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.217267 25709 raft_consensus.cc:2376] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2. Re-sending same reply.
W20260709 12:17:16.309947 26258 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.249s	user 0.007s	sys 0.004s
I20260709 12:17:16.311440 26250 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.050s
I20260709 12:17:16.325666 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:16.326323 25633 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 0.
W20260709 12:17:16.323767 25751 leader_election.cc:343] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.285328 26239 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.329471 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:16.329864 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:17:16.330866 25752 leader_election.cc:343] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.338321 25896 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.338809 25896 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.341279 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:16.328760 26237 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:16.336553 26231 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.424736 25827 tablet_bootstrap.cc:654] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.422871 26238 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.425072 25896 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:17:16.427944 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66337us
I20260709 12:17:16.428936 25882 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.429491 25882 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.430424 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:16.436030 25603 leader_election.cc:343] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.358654 26015 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.074s
W20260709 12:17:16.485105 25584 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:16.432235 26238 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.488765 25882 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:16.498050 25829 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:16.498764 25829 ts_tablet_manager.cc:1403] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.672s	user 0.048s	sys 0.040s
I20260709 12:17:16.490082 26238 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.503700 25829 raft_consensus.cc:359] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.508147 25829 raft_consensus.cc:385] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.508430 25829 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
W20260709 12:17:16.511619 26265 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:17:16.528537 26266 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 12:17:16.532831 26267 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:17:16.507894 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:16.533794 25634 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:16.532480 25829 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.559623 25677 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:16.568051 25882 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.568473 25882 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.568780 25882 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.572395 25783 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:16.577452 25882 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.584731 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:16.585448 25629 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.585338 26238 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.587182 25882 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:16.592535 25827 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:16.593017 25827 ts_tablet_manager.cc:1403] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.537s	user 0.019s	sys 0.001s
I20260709 12:17:16.594635 26234 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:17:16.599375 26239 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:16.603174 25829 ts_tablet_manager.cc:1434] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.104s	user 0.006s	sys 0.002s
I20260709 12:17:16.595863 25827 raft_consensus.cc:359] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.605020 25827 raft_consensus.cc:385] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.605376 25827 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.605551 25829 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Bootstrap starting.
I20260709 12:17:16.611045 25827 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.635504 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:16.636638 25705 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:16.641160 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:16.685622 25779 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:16.644486 25829 tablet_bootstrap.cc:654] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.652249 26238 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:16.684073 25827 ts_tablet_manager.cc:1434] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.001s
I20260709 12:17:16.694154 25827 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:16.648001 25629 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:17:16.715871 25677 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:16.707659 26234 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.716617 26234 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.719010 26234 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:16.716105 25829 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: No bootstrap required, opened a new log
I20260709 12:17:16.719892 26234 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.720822 26234 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:16.721935 26234 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.663290 25536 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 2, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.720014 25829 ts_tablet_manager.cc:1403] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Time spent bootstrapping tablet: real 0.115s	user 0.012s	sys 0.002s
I20260709 12:17:16.736059 25829 raft_consensus.cc:359] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.736789 25829 raft_consensus.cc:385] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.737077 25829 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.737744 25829 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.738500 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:16.739252 25629 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 0.
I20260709 12:17:16.740577 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:16.745010 25712 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.752066 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:16.752823 25712 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:16.752967 25779 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:16.755725 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:16.756699 26109 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.759693 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:16.760514 25785 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:16.763163 25677 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:16.764099 25882 raft_consensus.cc:2764] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:17:16.764964 26109 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.773129 26253 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.773656 26253 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.775774 26253 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:16.776124 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:16.777024 25785 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:17:16.778023 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:16.778270 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:16.779836 25533 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:16.780661 26237 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.781370 26237 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:16.782217 26237 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.794572 25827 tablet_bootstrap.cc:654] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:16.795106 25603 leader_election.cc:343] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.803365 25783 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:16.805323 25882 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:16.808380 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:16.808986 25783 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:16.809247 25535 catalog_manager.cc:5697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.824841 26251 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.863917 26251 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.890873 25829 ts_tablet_manager.cc:1434] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Time spent starting tablet: real 0.158s	user 0.008s	sys 0.003s
I20260709 12:17:16.907439 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:16.908325 25634 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 1: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:17:16.909507 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:16.919230 25713 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:16.927850 26249 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:16.928864 26251 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:16.932816 25827 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:16.933288 25827 ts_tablet_manager.cc:1403] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.239s	user 0.010s	sys 0.015s
I20260709 12:17:16.940693 25827 raft_consensus.cc:359] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.941349 25827 raft_consensus.cc:385] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.941633 25827 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.936661 25533 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 0 to 1, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:16.943615 25827 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:16.933645 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:16.953613 25706 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:16.951663 25629 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:17:16.953181 25827 ts_tablet_manager.cc:1434] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.003s
I20260709 12:17:16.954989 25603 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:16.955547 25827 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Bootstrap starting.
I20260709 12:17:16.956125 26249 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.969276 26249 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.969659 26249 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.981191 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:16.982038 25784 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 0.
I20260709 12:17:16.990688 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:16.996244 25896 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:16.985652 26238 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:17:16.996758 26244 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:17.002044 25601 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:17.009238 26146 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.012342 26146 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.012753 26146 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.020699 26249 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:17.023784 25827 tablet_bootstrap.cc:654] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.023999 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:17.024842 25705 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.026012 26146 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:17.043917 26249 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:17.066879 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:17.068645 25603 leader_election.cc:343] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.069881 26146 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:17.077118 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:17.077795 25785 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.080178 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:17.081138 25780 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.093376 25705 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:17.095242 25603 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:17.096020 26146 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:17.096658 26146 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:17.097506 26146 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:17.100044 25780 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:17.103207 25827 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: No bootstrap required, opened a new log
I20260709 12:17:17.104035 25827 ts_tablet_manager.cc:1403] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Time spent bootstrapping tablet: real 0.149s	user 0.029s	sys 0.022s
I20260709 12:17:17.105660 25601 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:17.109179 25827 raft_consensus.cc:359] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:17.109757 26156 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:17.109979 25827 raft_consensus.cc:385] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.110285 26156 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:17.110357 25827 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.111888 26156 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:17.112640 25785 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:17.113423 25827 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:17.115475 25827 ts_tablet_manager.cc:1434] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:17:17.128175 25533 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.149406 25533 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.157891 26234 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.160562 26234 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.161077 26234 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:17.170871 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:17.171203 26234 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:17.171438 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:17.172264 25781 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 1: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:17.173406 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:17.174463 26234 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.187230 25882 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.189895 26234 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:17.261379 26232 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.289784 26238 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.514837 26146 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.540570 26253 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.617012 26249 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.627549 25896 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.930465 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:17.960618 26298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:17.961011 26299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:17.963871 26301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:17.975250 24421 server_base.cc:1061] running on GCE node
I20260709 12:17:17.979552 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:17.979882 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:17.980160 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599437980129 us; error 0 us; skew 500 ppm
I20260709 12:17:17.980881 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:17.984983 24421 webserver.cc:533] Webserver started at http://127.23.217.68:40911/ using document root <none> and password file <none>
I20260709 12:17:17.985666 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:17.985904 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:17.986270 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:17.987624 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-3-root/instance:
uuid: "10d0c7349085463595aedaf7c071e742"
format_stamp: "Formatted at 2026-07-09 12:17:17 on dist-test-slave-lg8f"
I20260709 12:17:17.992774 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:17:17.998365 26306 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:17.999377 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:17:17.999748 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-3-root
uuid: "10d0c7349085463595aedaf7c071e742"
format_stamp: "Formatted at 2026-07-09 12:17:17 on dist-test-slave-lg8f"
I20260709 12:17:18.000069 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:18.022053 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:18.024814 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:18.028194 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:18.031638 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:18.031872 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:18.032245 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:18.032533 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:18.115790 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.68:35023
I20260709 12:17:18.116003 26368 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.68:35023 every 8 connection(s)
I20260709 12:17:18.130045 24421 auto_leader_rebalancer-test.cc:439] add a tserver: 10d0c7349085463595aedaf7c071e742
I20260709 12:17:18.148052 26369 heartbeater.cc:344] Connected to a master server at 127.23.217.126:36567
I20260709 12:17:18.148506 26369 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:18.149228 26369 heartbeater.cc:507] Master 127.23.217.126:36567 requested a full tablet report, sending...
I20260709 12:17:18.151183 25532 ts_manager.cc:194] Registered new tserver with Master: 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:18.152429 25532 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56312
I20260709 12:17:18.638139 25630 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:18.653851 25785 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:18.654369 25709 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:18.660195 26253 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:18.663525 26253 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:18.666281 25601 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b06b96578363435cbde83279e86e5c86. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:18.675004 26156 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.677089 25781 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.681351 25705 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.698132 25532 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.731101 25705 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:18.740176 25785 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:18.751647 25630 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:18.753069 25882 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:18.756338 25676 consensus_peers.cc:597] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 593fa615e9b64ba0974d4a40b5307c3c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:18.758741 26096 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:18.764669 26096 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.768198 25781 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.781352 25629 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.785904 25630 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:18.794375 25535 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.818785 25706 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:18.820444 25781 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:18.821446 25896 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.824694 26253 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.850445 26156 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.860980 25706 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:18.868468 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45814.5us
I20260709 12:17:18.879268 25706 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:18.887570 25533 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.889773 25785 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:18.895293 25601 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c30f8fbe9ff44588b4847921bab9bd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:18.914392 25785 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:18.915313 25676 consensus_peers.cc:597] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5735f590eff4fc9ae2686d09f8b6810. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:18.915865 25630 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:18.916332 25882 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.917341 26096 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.931012 26381 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.932560 25785 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.947808 25629 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:18.952337 25533 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:19.155303 26369 heartbeater.cc:499] Master 127.23.217.126:36567 was elected leader, sending a full tablet report...
I20260709 12:17:19.238683 26401 ts_tablet_manager.cc:933] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:19.243225 26401 tablet_copy_client.cc:323] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:19.264865 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet b06b96578363435cbde83279e86e5c86 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:19.265473 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet b06b96578363435cbde83279e86e5c86 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-b06b96578363435cbde83279e86e5c86
I20260709 12:17:19.273661 25648 tablet_copy_source_session.cc:215] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:19.279321 26401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b06b96578363435cbde83279e86e5c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:19.291309 26401 tablet_copy_client.cc:806] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:19.291926 26401 tablet_copy_client.cc:670] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:19.297482 26401 tablet_copy_client.cc:538] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:19.306627 26401 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:19.436857 26401 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:19.437863 26401 tablet_bootstrap.cc:492] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:19.438544 26401 ts_tablet_manager.cc:1403] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.132s	user 0.025s	sys 0.023s
I20260709 12:17:19.441943 26409 ts_tablet_manager.cc:933] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:19.446664 26401 raft_consensus.cc:359] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.447641 26401 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:19.448236 26401 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.452080 26401 ts_tablet_manager.cc:1434] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.013s	user 0.000s	sys 0.006s
I20260709 12:17:19.453588 26410 ts_tablet_manager.cc:933] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
W20260709 12:17:19.452456 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 83359.8us
I20260709 12:17:19.460412 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-b06b96578363435cbde83279e86e5c86 received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:19.460958 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-b06b96578363435cbde83279e86e5c86 on tablet b06b96578363435cbde83279e86e5c86 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:19.463944 26409 tablet_copy_client.cc:323] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:19.467162 26410 tablet_copy_client.cc:323] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:19.470026 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet 8c30f8fbe9ff44588b4847921bab9bd0 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:19.470598 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet 8c30f8fbe9ff44588b4847921bab9bd0 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-8c30f8fbe9ff44588b4847921bab9bd0
I20260709 12:17:19.477201 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 593fa615e9b64ba0974d4a40b5307c3c from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:19.477766 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 593fa615e9b64ba0974d4a40b5307c3c from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-593fa615e9b64ba0974d4a40b5307c3c
I20260709 12:17:19.478513 25648 tablet_copy_source_session.cc:215] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:19.482750 26410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c30f8fbe9ff44588b4847921bab9bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:19.484895 25723 tablet_copy_source_session.cc:215] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:19.488590 26409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593fa615e9b64ba0974d4a40b5307c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:19.503554 26410 tablet_copy_client.cc:806] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:19.504297 26410 tablet_copy_client.cc:670] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:19.504936 26409 tablet_copy_client.cc:806] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:19.505827 26409 tablet_copy_client.cc:670] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:19.508345 26410 tablet_copy_client.cc:538] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:19.509490 26409 tablet_copy_client.cc:538] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:19.516947 26409 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:19.517405 26410 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:19.541217 26401 ts_tablet_manager.cc:933] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:19.545346 26409 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:19.546456 26409 tablet_bootstrap.cc:492] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:19.547966 26409 ts_tablet_manager.cc:1403] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.010s
I20260709 12:17:19.555794 26410 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:19.558084 26410 tablet_bootstrap.cc:492] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:19.559296 26401 tablet_copy_client.cc:323] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:19.559975 26410 ts_tablet_manager.cc:1403] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.012s
I20260709 12:17:19.561486 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet e5735f590eff4fc9ae2686d09f8b6810 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:19.562067 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet e5735f590eff4fc9ae2686d09f8b6810 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-e5735f590eff4fc9ae2686d09f8b6810
I20260709 12:17:19.564673 26410 raft_consensus.cc:359] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.565655 26410 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:19.566174 26409 raft_consensus.cc:359] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.570034 26409 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:19.569356 26410 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.571573 26409 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.573365 26410 ts_tablet_manager.cc:1434] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:17:19.573997 26409 ts_tablet_manager.cc:1434] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.026s	user 0.000s	sys 0.006s
I20260709 12:17:19.575049 25723 tablet_copy_source_session.cc:215] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:19.575479 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-8c30f8fbe9ff44588b4847921bab9bd0 received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:19.576079 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-8c30f8fbe9ff44588b4847921bab9bd0 on tablet 8c30f8fbe9ff44588b4847921bab9bd0 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:19.578795 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-593fa615e9b64ba0974d4a40b5307c3c received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:19.580089 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-593fa615e9b64ba0974d4a40b5307c3c on tablet 593fa615e9b64ba0974d4a40b5307c3c with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:19.580138 26401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5735f590eff4fc9ae2686d09f8b6810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:19.596299 26401 tablet_copy_client.cc:806] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:19.596930 26401 tablet_copy_client.cc:670] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:19.600771 26401 tablet_copy_client.cc:538] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:19.610869 26401 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:19.630410 26401 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:19.631785 26401 tablet_bootstrap.cc:492] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:19.632416 26401 ts_tablet_manager.cc:1403] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 12:17:19.635399 26401 raft_consensus.cc:359] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.636024 26401 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:19.636518 26401 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:19.638706 26401 ts_tablet_manager.cc:1434] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:17:19.641577 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-e5735f590eff4fc9ae2686d09f8b6810 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:19.641942 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-e5735f590eff4fc9ae2686d09f8b6810 on tablet e5735f590eff4fc9ae2686d09f8b6810 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:19.702453 26344 raft_consensus.cc:1217] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:19.815683 26344 raft_consensus.cc:1217] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:19.933722 26344 raft_consensus.cc:1217] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:19.977174 26344 raft_consensus.cc:1217] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:20.264057 25896 raft_consensus.cc:1064] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:20.267215 25896 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.280718 26344 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.281508 25781 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.281840 25706 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.283182 26253 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.284606 26253 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.286439 25896 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.302981 26253 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.304320 26344 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.316498 25781 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.322757 25535 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.331533 25706 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.365906 26096 raft_consensus.cc:1064] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:20.369963 26096 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.377123 25634 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.386255 26344 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.387012 25706 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.388293 26156 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:20.390425 25896 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:20.395054 26344 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.395816 25780 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.396687 26400 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:20.397753 26399 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.404147 25896 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.405896 25709 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.408140 25634 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.409863 26399 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.418953 26344 raft_consensus.cc:2955] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.431969 26156 raft_consensus.cc:1064] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:20.440727 25533 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.456859 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b06b96578363435cbde83279e86e5c86 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:20.465051 26156 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.471036 26400 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.472700 25629 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.488931 26343 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.493439 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet b06b96578363435cbde83279e86e5c86 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:20.476121 25780 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.501828 26344 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.505537 25781 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.506196 25709 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.506614 26253 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.508834 26253 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.514582 26436 tablet_replica.cc:333] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:20.513995 25533 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:20.518006 26419 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:20.521371 26436 raft_consensus.cc:2243] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.522240 26436 raft_consensus.cc:2272] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.528801 26436 ts_tablet_manager.cc:1916] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:20.549259 26436 ts_tablet_manager.cc:1929] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:20.549788 26436 log.cc:1199] T b06b96578363435cbde83279e86e5c86 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/b06b96578363435cbde83279e86e5c86
I20260709 12:17:20.551952 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet b06b96578363435cbde83279e86e5c86 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:20.562954 25896 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.565896 26344 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.577935 25709 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.578357 25780 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.591230 25709 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.595170 26343 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.600906 25533 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.615067 26432 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:20.616506 26096 raft_consensus.cc:1064] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:20.622944 26399 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.623385 26096 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.586591 25781 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.635489 25629 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.640985 26344 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.642719 25896 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:20.643009 25781 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.644516 25896 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:20.646342 26344 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.651978 26432 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:20.653409 26433 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.654557 25781 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.657636 25780 raft_consensus.cc:2955] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.669932 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 593fa615e9b64ba0974d4a40b5307c3c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:20.670835 25629 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.672071 25785 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.673720 26400 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.674854 26343 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:20.675858 26432 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:20.680981 26432 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:20.679678 25535 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.687294 26342 raft_consensus.cc:2955] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.688721 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8c30f8fbe9ff44588b4847921bab9bd0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:20.699131 25532 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:20.711989 26400 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.721019 26343 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.734457 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c30f8fbe9ff44588b4847921bab9bd0 with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:20.745440 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 593fa615e9b64ba0974d4a40b5307c3c with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:20.748600 25780 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.762643 25634 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.768208 26446 tablet_replica.cc:333] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:20.769371 26446 raft_consensus.cc:2243] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.770220 26446 raft_consensus.cc:2272] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.773610 26445 tablet_replica.cc:333] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:20.780769 26446 ts_tablet_manager.cc:1916] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:20.784183 26445 raft_consensus.cc:2243] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.793828 25535 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.804351 26445 raft_consensus.cc:2272] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.826140 26446 ts_tablet_manager.cc:1929] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:20.830572 26446 log.cc:1199] T 593fa615e9b64ba0974d4a40b5307c3c P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/593fa615e9b64ba0974d4a40b5307c3c
I20260709 12:17:20.832470 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 593fa615e9b64ba0974d4a40b5307c3c (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:20.835768 26445 ts_tablet_manager.cc:1916] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:20.854908 25710 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:20.862121 26343 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.862234 25630 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:20.863812 26432 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:20.865096 26400 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:20.885363 26445 ts_tablet_manager.cc:1929] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:20.886063 26445 log.cc:1199] T 8c30f8fbe9ff44588b4847921bab9bd0 P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/8c30f8fbe9ff44588b4847921bab9bd0
I20260709 12:17:20.887907 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet 8c30f8fbe9ff44588b4847921bab9bd0 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:20.914776 25629 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.915043 26432 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.915151 26343 raft_consensus.cc:2955] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.930689 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e5735f590eff4fc9ae2686d09f8b6810 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:20.941242 25536 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:20.952366 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet e5735f590eff4fc9ae2686d09f8b6810 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:20.953411 26436 tablet_replica.cc:333] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:20.954380 26436 raft_consensus.cc:2243] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.955206 26436 raft_consensus.cc:2272] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.958175 26436 ts_tablet_manager.cc:1916] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:20.970299 26436 ts_tablet_manager.cc:1929] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:20.970763 26436 log.cc:1199] T e5735f590eff4fc9ae2686d09f8b6810 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/e5735f590eff4fc9ae2686d09f8b6810
I20260709 12:17:20.972355 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet e5735f590eff4fc9ae2686d09f8b6810 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:22.020001 25630 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.026634 25783 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:22.026710 25709 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:22.026899 25601 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e13025a828f141378027526ae4f0b002. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.028162 26448 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.028903 26434 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.040325 25709 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.037637 26448 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.042683 25783 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.059418 25532 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.070896 25783 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.079530 25709 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:22.079648 25630 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:22.081084 26407 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.081833 26447 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.102440 26447 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.107065 25629 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:22.111392 25750 consensus_peers.cc:597] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19825207f8744cf7809ab169c0ec8973. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.111388 25709 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.119522 25710 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.124403 25535 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:22.143491 25676 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 36db04d76d874d328966edd6be42e2c8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.144138 25783 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:22.145491 26399 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.147944 25629 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:22.151134 26400 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.169224 26465 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.171108 25630 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.172632 25780 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.188618 25532 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:22.184660 25780 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:17:22.217257 25750 consensus_peers.cc:597] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79cdf55909074ccd9e6b49679e14434d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.218333 25710 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:22.219575 25630 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:22.222220 26407 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.227435 26407 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.234153 26447 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.235954 25630 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.239575 25709 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.256119 25535 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:22.512630 26472 ts_tablet_manager.cc:933] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:22.514750 26472 tablet_copy_client.cc:323] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:22.516474 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet e13025a828f141378027526ae4f0b002 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:22.516944 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet e13025a828f141378027526ae4f0b002 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-e13025a828f141378027526ae4f0b002
I20260709 12:17:22.526438 25648 tablet_copy_source_session.cc:215] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:22.530007 26472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13025a828f141378027526ae4f0b002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.546878 26472 tablet_copy_client.cc:806] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:22.547583 26472 tablet_copy_client.cc:670] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:22.551174 26474 ts_tablet_manager.cc:933] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:22.551892 26472 tablet_copy_client.cc:538] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:22.555042 26474 tablet_copy_client.cc:323] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:22.565976 26472 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:22.580930 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet 19825207f8744cf7809ab169c0ec8973 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:22.581688 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet 19825207f8744cf7809ab169c0ec8973 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-19825207f8744cf7809ab169c0ec8973
I20260709 12:17:22.596962 26478 ts_tablet_manager.cc:933] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:22.600033 26472 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.601207 26472 tablet_bootstrap.cc:492] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:22.602123 26472 ts_tablet_manager.cc:1403] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.037s	user 0.027s	sys 0.000s
I20260709 12:17:22.606825 26472 raft_consensus.cc:359] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.607882 26472 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:22.608467 25797 tablet_copy_source_session.cc:215] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:22.608587 26472 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.611362 26478 tablet_copy_client.cc:323] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:22.612717 26474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19825207f8744cf7809ab169c0ec8973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.615572 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 36db04d76d874d328966edd6be42e2c8 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:22.616159 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 36db04d76d874d328966edd6be42e2c8 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-36db04d76d874d328966edd6be42e2c8
I20260709 12:17:22.626968 25723 tablet_copy_source_session.cc:215] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:22.631479 26472 ts_tablet_manager.cc:1434] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.029s	user 0.013s	sys 0.000s
I20260709 12:17:22.633440 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-e13025a828f141378027526ae4f0b002 received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:22.633883 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-e13025a828f141378027526ae4f0b002 on tablet e13025a828f141378027526ae4f0b002 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:22.639842 26478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36db04d76d874d328966edd6be42e2c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.660559 26478 tablet_copy_client.cc:806] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:22.661294 26478 tablet_copy_client.cc:670] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:22.661336 26474 tablet_copy_client.cc:806] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:22.662075 26474 tablet_copy_client.cc:670] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:22.665323 26478 tablet_copy_client.cc:538] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:22.666751 26474 tablet_copy_client.cc:538] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:22.674098 26478 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:22.676034 26474 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:22.702943 26474 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.706810 26474 tablet_bootstrap.cc:492] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:22.707472 26474 ts_tablet_manager.cc:1403] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.032s	user 0.025s	sys 0.000s
I20260709 12:17:22.713301 26474 raft_consensus.cc:359] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.714123 26474 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:22.714617 26478 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.714782 26474 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.715866 26478 tablet_bootstrap.cc:492] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:22.716645 26478 ts_tablet_manager.cc:1403] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.021s
I20260709 12:17:22.717267 26474 ts_tablet_manager.cc:1434] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.003s
I20260709 12:17:22.722247 26478 raft_consensus.cc:359] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.723166 26478 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:22.723847 26478 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.730535 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-19825207f8744cf7809ab169c0ec8973 received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:22.731062 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-19825207f8744cf7809ab169c0ec8973 on tablet 19825207f8744cf7809ab169c0ec8973 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:22.738209 26478 ts_tablet_manager.cc:1434] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:17:22.741482 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-36db04d76d874d328966edd6be42e2c8 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:22.741947 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-36db04d76d874d328966edd6be42e2c8 on tablet 36db04d76d874d328966edd6be42e2c8 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:22.764106 26478 ts_tablet_manager.cc:933] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:22.767776 26478 tablet_copy_client.cc:323] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:22.770444 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet 79cdf55909074ccd9e6b49679e14434d from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:22.771050 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet 79cdf55909074ccd9e6b49679e14434d from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-79cdf55909074ccd9e6b49679e14434d
I20260709 12:17:22.778474 25797 tablet_copy_source_session.cc:215] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:22.781512 26478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdf55909074ccd9e6b49679e14434d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.795212 26478 tablet_copy_client.cc:806] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:22.795861 26478 tablet_copy_client.cc:670] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:22.800015 26478 tablet_copy_client.cc:538] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:22.808871 26478 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:22.829809 26478 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.830751 26478 tablet_bootstrap.cc:492] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:22.831378 26478 ts_tablet_manager.cc:1403] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.007s
I20260709 12:17:22.834224 26478 raft_consensus.cc:359] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.835075 26478 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:22.835700 26478 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:22.837992 26478 ts_tablet_manager.cc:1434] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:22.839748 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-79cdf55909074ccd9e6b49679e14434d received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:22.840237 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-79cdf55909074ccd9e6b49679e14434d on tablet 79cdf55909074ccd9e6b49679e14434d with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:22.905747 26343 raft_consensus.cc:1217] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:22.982366 26343 raft_consensus.cc:1217] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:23.116788 26343 raft_consensus.cc:1217] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:23.158809 26343 raft_consensus.cc:1217] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:23.434263 26491 raft_consensus.cc:1064] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:23.440342 26491 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:23.454627 25709 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.456524 26343 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.456869 25781 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.459017 26419 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:23.460631 26490 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:23.461701 26493 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:23.485165 26484 raft_consensus.cc:1064] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:23.488821 26484 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:23.495893 26489 raft_consensus.cc:1064] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:23.499444 26489 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:23.522843 26342 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.525334 25636 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.526623 25780 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.527983 26468 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:23.529872 26468 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:23.536626 26489 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:23.557224 26486 raft_consensus.cc:1064] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:23.560616 26486 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:23.568307 26493 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.592958 26343 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.600519 26344 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.600598 25705 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.602901 26484 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:23.605001 26488 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:23.596925 25709 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.642215 25637 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.651629 26484 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:23.661866 26341 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.664840 26488 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:23.678073 26468 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.679854 25712 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.679965 25637 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.681699 26484 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:23.691851 25630 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:23.699880 26484 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:23.705435 25783 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.707762 25781 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.725406 25536 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.726150 26344 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.754238 25532 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:23.763648 26484 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.784204 25636 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.813468 26341 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:23.805716 26460 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:23.910125 26509 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:17:23.890740 26342 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:24.121171 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61559.8us
W20260709 12:17:24.203500 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 114783us
I20260709 12:17:24.218974 25706 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:24.220937 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55375.2us
I20260709 12:17:24.229548 25705 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.213310 25634 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.297868 25634 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:24.299786 25533 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.306594 25533 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.319634 26342 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.323541 25781 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.330945 26527 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:24.336205 26448 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:24.354822 26525 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.357488 26342 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.369987 25710 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:24.375629 25785 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:24.376426 25783 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:24.378940 25532 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.383903 25705 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.383915 26341 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.386214 26460 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:24.387123 26341 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.388849 25705 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.389798 26486 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:24.390635 26523 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:24.388970 26343 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.393882 26465 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:24.394516 25637 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:24.395092 26486 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:24.399650 26520 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:24.408747 26522 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.409605 25712 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.412381 26518 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.420233 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 79cdf55909074ccd9e6b49679e14434d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:24.422592 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet e13025a828f141378027526ae4f0b002 with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:24.427286 26342 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.432811 26341 raft_consensus.cc:2955] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.434434 26400 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.436831 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 19825207f8744cf7809ab169c0ec8973 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:24.436519 25710 raft_consensus.cc:2955] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.446069 26530 tablet_replica.cc:333] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:24.445379 25637 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.447395 25785 raft_consensus.cc:2955] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.450765 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 36db04d76d874d328966edd6be42e2c8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:24.469190 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e13025a828f141378027526ae4f0b002 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:24.451242 26343 raft_consensus.cc:2955] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.452519 25532 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:24.486703 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 79cdf55909074ccd9e6b49679e14434d with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:24.504518 26531 tablet_replica.cc:333] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:24.492372 25535 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.505940 26531 raft_consensus.cc:2243] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:24.506912 26531 raft_consensus.cc:2272] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:24.512102 26530 raft_consensus.cc:2243] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:24.512953 26530 raft_consensus.cc:2272] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:24.513056 26531 ts_tablet_manager.cc:1916] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:24.519196 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 36db04d76d874d328966edd6be42e2c8 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:24.525058 26530 ts_tablet_manager.cc:1916] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:24.542759 26532 tablet_replica.cc:333] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:24.542316 25532 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:24.544097 26532 raft_consensus.cc:2243] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:24.544996 26532 raft_consensus.cc:2272] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:24.551548 26531 ts_tablet_manager.cc:1929] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:24.552248 26531 log.cc:1199] T 79cdf55909074ccd9e6b49679e14434d P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/79cdf55909074ccd9e6b49679e14434d
I20260709 12:17:24.553149 26530 ts_tablet_manager.cc:1929] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:24.553699 26530 log.cc:1199] T e13025a828f141378027526ae4f0b002 P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/e13025a828f141378027526ae4f0b002
I20260709 12:17:24.554529 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 79cdf55909074ccd9e6b49679e14434d (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:24.557703 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 19825207f8744cf7809ab169c0ec8973 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:24.561837 26531 tablet_replica.cc:333] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:24.563033 26531 raft_consensus.cc:2243] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:24.563915 26531 raft_consensus.cc:2272] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:24.564880 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet e13025a828f141378027526ae4f0b002 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:24.563342 26532 ts_tablet_manager.cc:1916] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:24.570154 26531 ts_tablet_manager.cc:1916] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:24.587293 26532 ts_tablet_manager.cc:1929] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:24.587888 26532 log.cc:1199] T 36db04d76d874d328966edd6be42e2c8 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/36db04d76d874d328966edd6be42e2c8
I20260709 12:17:24.589421 26531 ts_tablet_manager.cc:1929] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:24.589953 26531 log.cc:1199] T 19825207f8744cf7809ab169c0ec8973 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/19825207f8744cf7809ab169c0ec8973
I20260709 12:17:24.591145 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 36db04d76d874d328966edd6be42e2c8 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:24.592314 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 19825207f8744cf7809ab169c0ec8973 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:25.636624 25634 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:25.641950 25710 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.642827 25781 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.643373 26525 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:25.647455 25601 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1ee3e27f67449aaab6c88508697a762. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:25.648613 26525 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.652698 26448 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.654876 25710 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.676465 25710 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:25.677111 25532 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.677635 25781 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.696132 25781 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.700510 25637 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.705046 26400 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:25.709709 26465 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:25.718338 25676 consensus_peers.cc:597] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ac10fecbdc4482e93572bd7ff87319e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:25.732508 26541 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.743997 25785 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.787645 25634 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:25.789055 25637 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:25.813086 25601 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e41fb20a5d340ab95f58b3ccc5c680f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:25.816864 25785 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.817373 25710 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.831092 26525 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.833465 26533 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.833056 25533 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.861761 26533 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.863477 25785 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.869477 25710 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.880486 25536 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.883330 25783 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:17:25.894479 25750 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb52f5946670453a8ec9b21d12ececbb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:25.895311 25638 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.895850 25710 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:25.896639 26460 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.897652 26484 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.908916 26460 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.917071 25710 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.924552 25638 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:25.955263 25536 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:26.072516 26555 ts_tablet_manager.cc:933] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:26.074975 26555 tablet_copy_client.cc:323] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:26.077126 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet b1ee3e27f67449aaab6c88508697a762 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:26.077762 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet b1ee3e27f67449aaab6c88508697a762 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-b1ee3e27f67449aaab6c88508697a762
I20260709 12:17:26.084983 25648 tablet_copy_source_session.cc:215] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:26.089046 26555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1ee3e27f67449aaab6c88508697a762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:26.103631 26555 tablet_copy_client.cc:806] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:26.104348 26555 tablet_copy_client.cc:670] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:26.109055 26555 tablet_copy_client.cc:538] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:26.116413 26555 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:26.157334 26558 ts_tablet_manager.cc:933] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:26.163966 26555 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.165189 26555 tablet_bootstrap.cc:492] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:26.165915 26555 ts_tablet_manager.cc:1403] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.050s	user 0.032s	sys 0.009s
I20260709 12:17:26.169174 26555 raft_consensus.cc:359] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.170048 26555 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:26.170859 26555 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.192060 26558 tablet_copy_client.cc:323] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:26.194382 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 5ac10fecbdc4482e93572bd7ff87319e from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:26.195013 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 5ac10fecbdc4482e93572bd7ff87319e from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-5ac10fecbdc4482e93572bd7ff87319e
I20260709 12:17:26.196400 26555 ts_tablet_manager.cc:1434] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.004s
I20260709 12:17:26.199290 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-b1ee3e27f67449aaab6c88508697a762 received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:26.199756 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-b1ee3e27f67449aaab6c88508697a762 on tablet b1ee3e27f67449aaab6c88508697a762 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:26.211570 25723 tablet_copy_source_session.cc:215] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:26.215478 26558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac10fecbdc4482e93572bd7ff87319e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:26.232453 26558 tablet_copy_client.cc:806] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:26.233155 26558 tablet_copy_client.cc:670] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:26.237350 26558 tablet_copy_client.cc:538] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:26.247787 26558 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:26.278473 26558 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.279453 26558 tablet_bootstrap.cc:492] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:26.280457 26558 ts_tablet_manager.cc:1403] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.015s
I20260709 12:17:26.284690 26558 raft_consensus.cc:359] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.285547 26558 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:26.286193 26558 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.289117 26558 ts_tablet_manager.cc:1434] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:17:26.291235 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-5ac10fecbdc4482e93572bd7ff87319e received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:26.291800 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-5ac10fecbdc4482e93572bd7ff87319e on tablet 5ac10fecbdc4482e93572bd7ff87319e with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:26.358668 26558 ts_tablet_manager.cc:933] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:26.360543 26558 tablet_copy_client.cc:323] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:26.362212 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet cb52f5946670453a8ec9b21d12ececbb from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:26.362769 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet cb52f5946670453a8ec9b21d12ececbb from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-cb52f5946670453a8ec9b21d12ececbb
I20260709 12:17:26.369299 25797 tablet_copy_source_session.cc:215] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:26.373708 26558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb52f5946670453a8ec9b21d12ececbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:26.391588 26558 tablet_copy_client.cc:806] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:26.392315 26558 tablet_copy_client.cc:670] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:26.398123 26558 tablet_copy_client.cc:538] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:26.409816 26558 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:26.427253 26558 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.428896 26558 tablet_bootstrap.cc:492] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:26.429546 26558 ts_tablet_manager.cc:1403] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 12:17:26.432327 26558 raft_consensus.cc:359] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.433296 26558 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:26.434067 26558 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.438081 26558 ts_tablet_manager.cc:1434] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:26.439885 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-cb52f5946670453a8ec9b21d12ececbb received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:26.440320 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-cb52f5946670453a8ec9b21d12ececbb on tablet cb52f5946670453a8ec9b21d12ececbb with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:26.446776 26558 ts_tablet_manager.cc:933] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:26.451894 26558 tablet_copy_client.cc:323] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:26.453462 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet 7e41fb20a5d340ab95f58b3ccc5c680f from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:26.453956 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet 7e41fb20a5d340ab95f58b3ccc5c680f from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-7e41fb20a5d340ab95f58b3ccc5c680f
I20260709 12:17:26.459916 25648 tablet_copy_source_session.cc:215] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:26.463161 26558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e41fb20a5d340ab95f58b3ccc5c680f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:26.484025 26558 tablet_copy_client.cc:806] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:26.484601 26558 tablet_copy_client.cc:670] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:26.492682 26558 tablet_copy_client.cc:538] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:26.500291 26558 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:26.517995 26558 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.519053 26558 tablet_bootstrap.cc:492] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:26.519750 26558 ts_tablet_manager.cc:1403] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.004s
I20260709 12:17:26.523272 26342 raft_consensus.cc:1217] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:26.523275 26558 raft_consensus.cc:359] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.526840 26558 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:26.527525 26558 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:26.530009 26558 ts_tablet_manager.cc:1434] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:17:26.534664 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-7e41fb20a5d340ab95f58b3ccc5c680f received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:26.535149 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-7e41fb20a5d340ab95f58b3ccc5c680f on tablet 7e41fb20a5d340ab95f58b3ccc5c680f with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:26.638833 26342 raft_consensus.cc:1217] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:26.832437 26342 raft_consensus.cc:1217] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:26.942301 26342 raft_consensus.cc:1217] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:26.969053 26448 raft_consensus.cc:1064] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:26.971815 26448 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:26.984644 25785 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:26.984644 26342 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:26.987063 25710 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:26.986888 26567 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:26.988045 26448 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:26.988888 26525 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.010932 26533 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.022588 25785 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.021886 26342 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.039088 25533 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:27.054204 25710 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.085685 25638 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:27.093670 26342 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:27.095093 26533 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:27.095661 25785 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:27.099103 26567 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:27.107829 26567 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.113229 26342 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.117211 25783 raft_consensus.cc:2955] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.125938 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b1ee3e27f67449aaab6c88508697a762 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:27.127991 25535 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.143282 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet b1ee3e27f67449aaab6c88508697a762 with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:27.146953 26576 tablet_replica.cc:333] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:27.154318 26576 raft_consensus.cc:2243] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:27.155331 26576 raft_consensus.cc:2272] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:27.159204 26576 ts_tablet_manager.cc:1916] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:27.176883 26576 ts_tablet_manager.cc:1929] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:27.177299 26576 log.cc:1199] T b1ee3e27f67449aaab6c88508697a762 P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/b1ee3e27f67449aaab6c88508697a762
I20260709 12:17:27.178819 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet b1ee3e27f67449aaab6c88508697a762 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:27.214145 26541 raft_consensus.cc:1064] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:27.216956 26541 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:27.226910 26342 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.229218 26542 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.229964 25783 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.231706 26541 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.238348 25634 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.245077 26542 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.286594 26542 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.291790 25780 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.302850 25634 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.311579 25536 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.322410 26342 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.352242 25710 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:27.359190 26342 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:27.359763 25780 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:27.360523 26542 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:27.362216 26566 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:27.384771 26448 raft_consensus.cc:1064] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:27.385670 26541 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.387156 26448 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:27.387936 25783 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.401464 26343 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.402659 25785 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.404752 26533 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.406613 26533 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.407985 26343 raft_consensus.cc:2955] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.409330 25535 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.418040 25712 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.420674 26525 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.436640 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ac10fecbdc4482e93572bd7ff87319e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:27.444978 26533 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.448452 25705 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.452049 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ac10fecbdc4482e93572bd7ff87319e with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:27.464437 26341 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.468639 25780 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.483371 25535 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.489022 26586 tablet_replica.cc:333] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:27.490245 26586 raft_consensus.cc:2243] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:27.491048 26586 raft_consensus.cc:2272] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:27.502535 26460 raft_consensus.cc:1064] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:27.505208 26460 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:27.510193 25634 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:27.511453 26586 ts_tablet_manager.cc:1916] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:27.524278 26341 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:27.526641 25705 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:27.530860 26448 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:27.532151 26533 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:27.540853 26586 ts_tablet_manager.cc:1929] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:27.541306 26586 log.cc:1199] T 5ac10fecbdc4482e93572bd7ff87319e P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/5ac10fecbdc4482e93572bd7ff87319e
I20260709 12:17:27.540485 26525 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.544103 25705 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.550889 26343 raft_consensus.cc:2955] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.560141 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 5ac10fecbdc4482e93572bd7ff87319e (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:27.562067 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7e41fb20a5d340ab95f58b3ccc5c680f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:27.562108 25532 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.591935 26343 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.596406 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e41fb20a5d340ab95f58b3ccc5c680f with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:27.602650 26460 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:27.609992 25712 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.612501 26589 tablet_replica.cc:333] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:27.614928 26585 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:27.623438 25634 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:27.634753 26587 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.637578 25712 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.641796 26484 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:27.649549 26341 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.663224 25532 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:27.667959 25634 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:27.694265 26589 raft_consensus.cc:2243] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:27.695415 26589 raft_consensus.cc:2272] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:27.700104 26589 ts_tablet_manager.cc:1916] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:27.727691 26589 ts_tablet_manager.cc:1929] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:27.728272 26589 log.cc:1199] T 7e41fb20a5d340ab95f58b3ccc5c680f P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/7e41fb20a5d340ab95f58b3ccc5c680f
I20260709 12:17:27.730086 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 7e41fb20a5d340ab95f58b3ccc5c680f (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:28.213054 25785 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:28.219765 26341 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:28.221143 25634 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:28.222643 26484 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:28.225988 26587 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:28.234122 26484 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:28.243937 25634 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:28.246701 26341 raft_consensus.cc:2955] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:28.264606 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cb52f5946670453a8ec9b21d12ececbb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:28.265874 25532 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:28.288870 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet cb52f5946670453a8ec9b21d12ececbb with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:28.290977 26599 tablet_replica.cc:333] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:28.291915 26599 raft_consensus.cc:2243] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:28.292563 26599 raft_consensus.cc:2272] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:28.295815 26599 ts_tablet_manager.cc:1916] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:28.310643 26599 ts_tablet_manager.cc:1929] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:28.310961 26599 log.cc:1199] T cb52f5946670453a8ec9b21d12ececbb P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/cb52f5946670453a8ec9b21d12ececbb
I20260709 12:17:28.312367 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet cb52f5946670453a8ec9b21d12ececbb (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:29.361387 25705 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:17:29.368917 25676 consensus_peers.cc:597] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9eec1cb847cc4587be1133c287c515b2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:29.370548 25634 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:29.370471 25783 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:29.372864 26601 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:29.374770 26603 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.382596 26542 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.384317 25634 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.384547 25785 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.400214 25536 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:29.403786 25785 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:17:29.415081 25750 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f6cc82bb18354fbf8395678374e211b2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:29.415563 25705 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:29.417141 26484 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.417273 25634 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:29.419808 26593 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.430123 26484 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.432791 25705 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.445513 25634 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.447280 25637 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:29.461248 25705 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:29.461637 25601 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb1874a5bbce42ad864b2ba3624a0c1c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:29.462704 25785 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:29.462764 26525 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:17:29.467240 26525 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.472373 25535 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:29.489184 26600 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.493914 25705 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.495818 25780 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.519668 25712 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:29.531282 25532 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:29.545464 25676 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78bc05d6a56b407bb7507c25a641dc4e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:29.549263 25785 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:29.550625 26542 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.554713 25634 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:29.556135 26542 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.567680 26601 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.569986 25785 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.571135 25634 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.594925 25535 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:29.823185 26621 ts_tablet_manager.cc:933] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:29.828862 26621 tablet_copy_client.cc:323] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:29.830930 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 9eec1cb847cc4587be1133c287c515b2 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:29.831523 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 9eec1cb847cc4587be1133c287c515b2 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-9eec1cb847cc4587be1133c287c515b2
I20260709 12:17:29.838964 25723 tablet_copy_source_session.cc:215] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:29.843585 26621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eec1cb847cc4587be1133c287c515b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:29.861083 26621 tablet_copy_client.cc:806] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:29.861867 26621 tablet_copy_client.cc:670] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:29.866055 26621 tablet_copy_client.cc:538] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:29.875401 26621 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:29.882418 26623 ts_tablet_manager.cc:933] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:29.888527 26623 tablet_copy_client.cc:323] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:29.891562 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet f6cc82bb18354fbf8395678374e211b2 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:29.892455 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet f6cc82bb18354fbf8395678374e211b2 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-f6cc82bb18354fbf8395678374e211b2
I20260709 12:17:29.904608 26621 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.905671 26621 tablet_bootstrap.cc:492] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:29.906379 26621 ts_tablet_manager.cc:1403] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.008s
I20260709 12:17:29.909550 26621 raft_consensus.cc:359] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:29.910348 26621 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:29.910813 25797 tablet_copy_source_session.cc:215] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:29.911038 26621 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:29.919133 26623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6cc82bb18354fbf8395678374e211b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:29.920660 26621 ts_tablet_manager.cc:1434] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.004s
I20260709 12:17:29.922667 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-9eec1cb847cc4587be1133c287c515b2 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:29.923101 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-9eec1cb847cc4587be1133c287c515b2 on tablet 9eec1cb847cc4587be1133c287c515b2 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:29.941669 26623 tablet_copy_client.cc:806] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:29.942276 26623 tablet_copy_client.cc:670] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:29.946475 26623 tablet_copy_client.cc:538] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:29.953455 26623 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:29.968655 26621 ts_tablet_manager.cc:933] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:29.971459 26621 tablet_copy_client.cc:323] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:29.973690 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet fb1874a5bbce42ad864b2ba3624a0c1c from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:29.974304 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet fb1874a5bbce42ad864b2ba3624a0c1c from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-fb1874a5bbce42ad864b2ba3624a0c1c
I20260709 12:17:29.976680 26623 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.977715 26623 tablet_bootstrap.cc:492] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:29.978439 26623 ts_tablet_manager.cc:1403] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.001s
I20260709 12:17:29.981787 26623 raft_consensus.cc:359] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:29.982642 26623 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:29.983130 25648 tablet_copy_source_session.cc:215] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:29.983544 26623 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:29.987252 26621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb1874a5bbce42ad864b2ba3624a0c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:29.988447 26623 ts_tablet_manager.cc:1434] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:17:29.991441 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-f6cc82bb18354fbf8395678374e211b2 received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:29.991876 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-f6cc82bb18354fbf8395678374e211b2 on tablet f6cc82bb18354fbf8395678374e211b2 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:30.008083 26621 tablet_copy_client.cc:806] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:30.008688 26621 tablet_copy_client.cc:670] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:30.012401 26623 ts_tablet_manager.cc:933] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:30.013495 26621 tablet_copy_client.cc:538] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:30.014478 26623 tablet_copy_client.cc:323] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:30.016165 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 78bc05d6a56b407bb7507c25a641dc4e from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:30.016657 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 78bc05d6a56b407bb7507c25a641dc4e from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-78bc05d6a56b407bb7507c25a641dc4e
I20260709 12:17:30.024128 26621 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:30.032028 25723 tablet_copy_source_session.cc:215] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:30.042008 26623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78bc05d6a56b407bb7507c25a641dc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:30.071182 26623 tablet_copy_client.cc:806] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:30.071830 26623 tablet_copy_client.cc:670] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:30.077821 26623 tablet_copy_client.cc:538] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:30.082314 26621 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:30.083456 26621 tablet_bootstrap.cc:492] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:30.084251 26621 ts_tablet_manager.cc:1403] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.060s	user 0.020s	sys 0.004s
I20260709 12:17:30.099596 26621 raft_consensus.cc:359] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:30.100668 26621 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:30.104841 26623 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:30.101385 26621 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:30.110352 26621 ts_tablet_manager.cc:1434] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:17:30.115093 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-fb1874a5bbce42ad864b2ba3624a0c1c received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:30.115615 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-fb1874a5bbce42ad864b2ba3624a0c1c on tablet fb1874a5bbce42ad864b2ba3624a0c1c with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:30.144034 26623 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:30.145066 26623 tablet_bootstrap.cc:492] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:30.145809 26623 ts_tablet_manager.cc:1403] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.041s	user 0.026s	sys 0.011s
I20260709 12:17:30.148953 26623 raft_consensus.cc:359] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:30.149823 26623 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:30.150462 26623 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:30.153280 26623 ts_tablet_manager.cc:1434] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:17:30.155126 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-78bc05d6a56b407bb7507c25a641dc4e received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:30.155548 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-78bc05d6a56b407bb7507c25a641dc4e on tablet 78bc05d6a56b407bb7507c25a641dc4e with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:30.256536 26341 raft_consensus.cc:1217] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:30.419276 26341 raft_consensus.cc:1217] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:30.475348 26341 raft_consensus.cc:1217] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:30.528151 26341 raft_consensus.cc:1217] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:30.850286 26484 raft_consensus.cc:1064] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:30.852620 26484 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:30.860642 26341 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:30.863255 26608 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:30.871505 25705 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:30.872160 25634 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:30.873245 26484 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:30.875116 26593 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:30.892923 26593 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:30.925096 25712 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:30.939253 26635 raft_consensus.cc:1064] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:30.971371 26635 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:30.976575 26601 raft_consensus.cc:1064] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
W20260709 12:17:30.978305 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 104090us
I20260709 12:17:30.981853 25536 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:31.008498 25629 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.010066 26634 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.025369 26601 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:31.033062 26632 raft_consensus.cc:1064] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:31.040933 25634 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:31.031256 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 64949.7us
I20260709 12:17:31.045238 26343 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.046128 25782 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.052309 26632 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:31.055285 25638 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.056695 26343 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.056864 26634 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:31.063102 26542 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.064934 26542 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.078593 26634 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:31.091961 25782 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.094678 26644 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:31.102777 26342 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.105046 25785 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.106335 26645 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:31.108805 26635 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.110862 26601 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.114214 26617 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:31.119637 26343 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.125537 25712 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:31.124953 26342 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.127226 26633 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.140549 25634 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.177155 25532 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:31.171864 25638 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.191547 25532 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:31.212977 25780 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.215066 26632 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.217496 26341 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.233014 25712 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.252169 25710 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:31.291185 26343 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.292821 26653 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:31.295650 25638 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.303779 26542 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.305835 26343 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.334414 25532 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.339200 25782 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.348495 25780 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.350530 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 78bc05d6a56b407bb7507c25a641dc4e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:31.364431 25710 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:31.377632 26542 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:31.382548 25536 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.387137 25634 raft_consensus.cc:2955] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.394387 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 78bc05d6a56b407bb7507c25a641dc4e with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:31.403962 26666 tablet_replica.cc:333] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:31.405190 26666 raft_consensus.cc:2243] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.405948 26666 raft_consensus.cc:2272] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.386001 25638 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:31.418761 25780 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.420661 26653 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:31.423870 26341 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.425195 26653 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:31.422715 25710 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.421059 26666 ts_tablet_manager.cc:1916] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:31.431838 26665 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:31.436728 26343 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.438189 26665 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:31.446760 26617 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.451623 26666 ts_tablet_manager.cc:1929] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:31.452235 26666 log.cc:1199] T 78bc05d6a56b407bb7507c25a641dc4e P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/78bc05d6a56b407bb7507c25a641dc4e
I20260709 12:17:31.454229 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 78bc05d6a56b407bb7507c25a641dc4e (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:31.445523 26634 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.461853 26343 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.468585 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9eec1cb847cc4587be1133c287c515b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:31.469455 25780 raft_consensus.cc:2955] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.471810 26341 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.475901 25535 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:31.479375 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb1874a5bbce42ad864b2ba3624a0c1c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:31.494359 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 9eec1cb847cc4587be1133c287c515b2 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:31.502091 25533 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.504235 26667 tablet_replica.cc:333] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:31.509848 25712 raft_consensus.cc:2955] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.508534 26341 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.520192 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet fb1874a5bbce42ad864b2ba3624a0c1c with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:31.521526 26666 tablet_replica.cc:333] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:31.525583 26666 raft_consensus.cc:2243] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.526378 26666 raft_consensus.cc:2272] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.530457 26666 ts_tablet_manager.cc:1916] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:31.553259 26667 raft_consensus.cc:2243] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.567019 26667 raft_consensus.cc:2272] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.569061 25780 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:31.571329 26666 ts_tablet_manager.cc:1929] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:31.571882 26666 log.cc:1199] T fb1874a5bbce42ad864b2ba3624a0c1c P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/fb1874a5bbce42ad864b2ba3624a0c1c
I20260709 12:17:31.573428 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet fb1874a5bbce42ad864b2ba3624a0c1c (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:31.575269 26667 ts_tablet_manager.cc:1916] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:31.577401 26341 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.578845 25634 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:31.579037 26608 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:31.580799 26642 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:31.591154 26667 ts_tablet_manager.cc:1929] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:31.591718 26667 log.cc:1199] T 9eec1cb847cc4587be1133c287c515b2 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/9eec1cb847cc4587be1133c287c515b2
I20260709 12:17:31.593482 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 9eec1cb847cc4587be1133c287c515b2 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:31.594735 26641 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.599113 26341 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.606844 25634 raft_consensus.cc:2955] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:31.612870 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f6cc82bb18354fbf8395678374e211b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:31.628921 25532 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:31.644470 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet f6cc82bb18354fbf8395678374e211b2 with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:31.651273 26668 tablet_replica.cc:333] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:31.652477 26668 raft_consensus.cc:2243] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.653301 26668 raft_consensus.cc:2272] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.657429 26668 ts_tablet_manager.cc:1916] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:31.673082 26668 ts_tablet_manager.cc:1929] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:31.673434 26668 log.cc:1199] T f6cc82bb18354fbf8395678374e211b2 P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/f6cc82bb18354fbf8395678374e211b2
I20260709 12:17:31.674975 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet f6cc82bb18354fbf8395678374e211b2 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:32.730311 25780 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:32.736198 25634 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:32.736456 25750 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9052f9b88ca4b9ebb24b8764ad345af. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:32.737753 26672 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.738125 25712 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:32.739785 26672 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:32.749339 26593 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.751042 25634 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.754537 25712 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.767979 25780 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:32.768105 25532 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:32.775243 25750 consensus_peers.cc:597] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a760df354a14284b004635d45d88a3e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:32.778067 25712 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:32.778260 25634 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:32.779587 26593 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.780339 26672 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.796288 26642 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.798458 25712 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.813966 25634 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.842324 25638 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:32.847149 25533 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.853168 25780 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:32.854727 26675 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.858230 25712 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:32.859665 26674 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:32.860468 25601 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a039e6df55747e1b662ecb5d069264b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:32.864519 26669 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.870862 25785 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.889374 25711 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:32.893049 25710 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.896728 25785 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:32.897794 25676 consensus_peers.cc:597] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b97aeee11fff4d52bb8252149577f56e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:32.898209 25638 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:32.899362 26601 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.900753 26601 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.903354 25532 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.917913 26671 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.930402 25638 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.935091 25785 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:32.947450 25534 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:33.273147 26695 ts_tablet_manager.cc:933] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:33.276048 26695 tablet_copy_client.cc:323] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:33.278173 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet b9052f9b88ca4b9ebb24b8764ad345af from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:33.278680 26697 ts_tablet_manager.cc:933] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:33.278728 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet b9052f9b88ca4b9ebb24b8764ad345af from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-b9052f9b88ca4b9ebb24b8764ad345af
I20260709 12:17:33.293526 25797 tablet_copy_source_session.cc:215] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:33.297902 26697 tablet_copy_client.cc:323] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:33.299019 26695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9052f9b88ca4b9ebb24b8764ad345af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:33.300025 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet 3a039e6df55747e1b662ecb5d069264b from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:33.300554 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet 3a039e6df55747e1b662ecb5d069264b from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-3a039e6df55747e1b662ecb5d069264b
I20260709 12:17:33.317101 25648 tablet_copy_source_session.cc:215] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:33.317533 26695 tablet_copy_client.cc:806] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:33.318181 26695 tablet_copy_client.cc:670] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:33.320472 26697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a039e6df55747e1b662ecb5d069264b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:33.326196 26695 tablet_copy_client.cc:538] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:33.335847 26695 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:33.342067 26697 tablet_copy_client.cc:806] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:33.390640 26697 tablet_copy_client.cc:670] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:33.421660 26697 tablet_copy_client.cc:538] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:33.448257 26697 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:33.489719 26701 ts_tablet_manager.cc:933] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:33.553051 26701 tablet_copy_client.cc:323] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:33.553853 26695 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:17:33.554731 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 71008.7us
I20260709 12:17:33.555269 26695 tablet_bootstrap.cc:492] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:33.556478 26695 ts_tablet_manager.cc:1403] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.221s	user 0.027s	sys 0.011s
I20260709 12:17:33.560678 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet 5a760df354a14284b004635d45d88a3e from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:33.561733 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet 5a760df354a14284b004635d45d88a3e from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-5a760df354a14284b004635d45d88a3e
I20260709 12:17:33.614758 26695 raft_consensus.cc:359] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:33.616953 26695 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:33.617925 26695 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:17:33.619843 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 93467.9us
I20260709 12:17:33.626924 25797 tablet_copy_source_session.cc:215] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:33.638244 26701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a760df354a14284b004635d45d88a3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:33.639130 26695 ts_tablet_manager.cc:1434] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.082s	user 0.011s	sys 0.005s
I20260709 12:17:33.654122 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-b9052f9b88ca4b9ebb24b8764ad345af received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:33.654837 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-b9052f9b88ca4b9ebb24b8764ad345af on tablet b9052f9b88ca4b9ebb24b8764ad345af with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:33.666694 26695 ts_tablet_manager.cc:933] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:33.683264 26701 tablet_copy_client.cc:806] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:33.703591 26701 tablet_copy_client.cc:670] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:33.720080 26695 tablet_copy_client.cc:323] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:33.736804 26701 tablet_copy_client.cc:538] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:33.747071 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet b97aeee11fff4d52bb8252149577f56e from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:33.747946 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet b97aeee11fff4d52bb8252149577f56e from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-b97aeee11fff4d52bb8252149577f56e
I20260709 12:17:33.749976 26697 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:33.751377 26697 tablet_bootstrap.cc:492] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:33.753063 26697 ts_tablet_manager.cc:1403] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.305s	user 0.015s	sys 0.019s
I20260709 12:17:33.784893 26697 raft_consensus.cc:359] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:33.785600 26697 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:33.786149 26697 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:33.788123 26697 ts_tablet_manager.cc:1434] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 12:17:33.790875 25723 tablet_copy_source_session.cc:215] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:33.766681 26701 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:33.798962 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-3a039e6df55747e1b662ecb5d069264b received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:33.799599 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-3a039e6df55747e1b662ecb5d069264b on tablet 3a039e6df55747e1b662ecb5d069264b with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:33.806782 26343 raft_consensus.cc:1217] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:33.812896 26695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97aeee11fff4d52bb8252149577f56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:33.863057 26695 tablet_copy_client.cc:806] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:33.863850 26695 tablet_copy_client.cc:670] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:33.868767 26343 raft_consensus.cc:1217] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:33.888836 26695 tablet_copy_client.cc:538] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:33.906764 26695 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:33.945101 26701 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:33.946239 26701 tablet_bootstrap.cc:492] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:33.947023 26701 ts_tablet_manager.cc:1403] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.190s	user 0.059s	sys 0.063s
I20260709 12:17:33.951802 26701 raft_consensus.cc:359] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:33.952654 26701 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:33.953362 26701 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:33.955860 26701 ts_tablet_manager.cc:1434] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:33.963506 26695 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:33.965441 26695 tablet_bootstrap.cc:492] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:33.965752 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-5a760df354a14284b004635d45d88a3e received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:33.966190 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-5a760df354a14284b004635d45d88a3e on tablet 5a760df354a14284b004635d45d88a3e with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:33.966192 26695 ts_tablet_manager.cc:1403] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.060s	user 0.021s	sys 0.007s
I20260709 12:17:33.972646 26695 raft_consensus.cc:359] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:33.973735 26695 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:33.974417 26695 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:33.977626 26695 ts_tablet_manager.cc:1434] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:17:33.980827 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-b97aeee11fff4d52bb8252149577f56e received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:33.981379 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-b97aeee11fff4d52bb8252149577f56e on tablet b97aeee11fff4d52bb8252149577f56e with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:33.981607 26343 raft_consensus.cc:1217] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:34.120960 26343 raft_consensus.cc:1217] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:34.240366 26673 raft_consensus.cc:1064] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:34.244199 26673 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.252967 25710 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.255358 26673 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.263777 25780 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.264017 26343 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.267050 26712 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.270862 26711 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.276126 26712 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.279093 26343 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.290529 25712 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.292718 25780 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.303926 25532 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:34.361245 25636 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.368587 26343 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.368839 25780 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.370177 26712 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:34.371416 26713 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:34.384709 25780 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.392117 26343 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.388505 26673 raft_consensus.cc:2955] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.411803 25533 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.428301 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a039e6df55747e1b662ecb5d069264b with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:34.432092 26725 tablet_replica.cc:333] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:34.433669 26725 raft_consensus.cc:2243] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.433724 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3a039e6df55747e1b662ecb5d069264b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:34.434818 26725 raft_consensus.cc:2272] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.440402 26725 ts_tablet_manager.cc:1916] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:34.468261 26725 ts_tablet_manager.cc:1929] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:34.472280 26725 log.cc:1199] T 3a039e6df55747e1b662ecb5d069264b P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/3a039e6df55747e1b662ecb5d069264b
I20260709 12:17:34.473038 26672 raft_consensus.cc:1064] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:34.475220 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet 3a039e6df55747e1b662ecb5d069264b (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:34.475936 26672 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.492895 26702 raft_consensus.cc:1064] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:34.496435 26702 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.516004 26341 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.517887 25712 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.519783 26727 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:34.522601 26686 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.532486 25636 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.534130 26686 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:34.538949 26341 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.539426 25636 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.549220 26732 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.550390 25710 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.550865 26702 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.558338 26686 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.604368 26732 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:34.634927 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 93323.3us
I20260709 12:17:34.645098 26341 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.647686 26702 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.666905 25710 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.672749 25637 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.678036 25711 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.680569 25532 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:34.694125 26341 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.705085 25634 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.711720 26671 raft_consensus.cc:1064] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:34.730371 26671 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.735481 25534 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.746013 25780 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.756735 26341 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.760327 26705 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.767988 26341 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.769719 26733 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:34.772739 25710 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.775861 26702 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:34.787645 26343 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.793932 25780 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.797492 26741 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.801383 25710 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.809096 26732 raft_consensus.cc:2955] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.815416 25785 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.816612 25634 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:34.819897 26671 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:34.822508 25533 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.830094 26343 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.829788 26671 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.842056 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5a760df354a14284b004635d45d88a3e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:34.882660 26341 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.885902 26702 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:17:34.891564 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 43494.2us
I20260709 12:17:34.896363 25634 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.898413 25785 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.901640 26727 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:34.904752 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a760df354a14284b004635d45d88a3e with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:34.906055 26343 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.923511 26733 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.931272 25636 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.935252 25634 raft_consensus.cc:2955] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.948978 25532 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.959725 26746 tablet_replica.cc:333] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:34.972713 26746 raft_consensus.cc:2243] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.973506 26746 raft_consensus.cc:2272] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.976280 25711 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:34.976444 25532 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:34.988754 26343 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.991472 25780 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:34.991627 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b9052f9b88ca4b9ebb24b8764ad345af with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:34.993052 26743 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:34.995015 26746 ts_tablet_manager.cc:1916] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:35.001232 26741 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:35.004755 26671 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:35.021962 26746 ts_tablet_manager.cc:1929] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:35.022539 26746 log.cc:1199] T 5a760df354a14284b004635d45d88a3e P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/5a760df354a14284b004635d45d88a3e
I20260709 12:17:35.024343 26343 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:35.026516 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b97aeee11fff4d52bb8252149577f56e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:35.029368 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet b9052f9b88ca4b9ebb24b8764ad345af with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:35.030994 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 5a760df354a14284b004635d45d88a3e (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:35.036638 25780 raft_consensus.cc:2955] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:35.038668 25532 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:35.056789 26747 tablet_replica.cc:333] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:35.057912 26747 raft_consensus.cc:2243] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:35.059937 26747 raft_consensus.cc:2272] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:35.067771 26747 ts_tablet_manager.cc:1916] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:35.069068 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet b97aeee11fff4d52bb8252149577f56e with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:35.070472 26746 tablet_replica.cc:333] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:35.071589 26746 raft_consensus.cc:2243] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:35.072340 26746 raft_consensus.cc:2272] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:35.080713 26746 ts_tablet_manager.cc:1916] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:35.089890 26747 ts_tablet_manager.cc:1929] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:35.090662 26747 log.cc:1199] T b9052f9b88ca4b9ebb24b8764ad345af P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/b9052f9b88ca4b9ebb24b8764ad345af
I20260709 12:17:35.093870 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet b9052f9b88ca4b9ebb24b8764ad345af (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:35.098256 26746 ts_tablet_manager.cc:1929] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:35.098759 26746 log.cc:1199] T b97aeee11fff4d52bb8252149577f56e P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/b97aeee11fff4d52bb8252149577f56e
I20260709 12:17:35.100528 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet b97aeee11fff4d52bb8252149577f56e (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
W20260709 12:17:37.066339 25750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.000s	sys 0.020s
W20260709 12:17:37.066766 25750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.001s	sys 0.020s
I20260709 12:17:37.074062 26713 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:37.074939 26713 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.077517 26713 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:17:37.105468 25753 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.024s	sys 0.005s
W20260709 12:17:37.105923 25753 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.024s	sys 0.005s
I20260709 12:17:37.135569 26751 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:37.136147 26751 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.138537 26753 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:37.145150 26752 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:37.171147 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.170660 26753 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.188738 26672 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:17:37.204166 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55043.2us
I20260709 12:17:37.186780 26752 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.204468 25536 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 197) took 1025 ms. Trace:
I20260709 12:17:37.218319 26752 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.220991 26769 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:37.221675 26769 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.229283 26753 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.205363 26672 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:37.230403 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 57196.3us
I20260709 12:17:37.232429 26672 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.212482 25536 rpcz_store.cc:276] 0709 12:17:36.179253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.179934 (+   681us) service_pool.cc:224] Handling call
0709 12:17:37.204255 (+1024321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21321,"reported_tablets":3}
I20260709 12:17:37.214740 26751 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.259898 26769 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.262111 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:37.266621 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60842.1us
I20260709 12:17:37.270174 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.273669 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.275259 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.278672 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.284096 26774 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:37.284870 26774 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.287122 26774 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.288271 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.291059 26753 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:37.291687 26753 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.293897 26753 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.295439 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.295483 26755 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:37.296532 26755 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:37.300103 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.303011 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:37.303797 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.304629 26755 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.309257 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.311501 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.313666 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.315587 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.316780 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.317958 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:37.317276 25751 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.319995 26775 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.320638 25751 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.321593 26775 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.322196 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.323099 26775 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.323472 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.324282 26775 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.324623 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.325038 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:37.325448 26775 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.325809 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.326906 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.327073 26775 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.327618 26755 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.328814 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.329527 26755 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.340536 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:37.341545 26772 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.352327 25709 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:37.376345 25783 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:37.376353 25636 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:37.377127 25676 consensus_peers.cc:597] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 402bc370028b4bc097d5c2fb4342cdfa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:37.378343 26771 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.379305 26705 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.389214 26705 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.391153 25785 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.405328 25780 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:37.413316 25536 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:37.423022 25636 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:37.437443 25750 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a741c75d1e684375b51f415fd0aa0296. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:37.441972 25709 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:37.443641 26755 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.444386 25636 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:37.447064 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08001a7ce0 after lost signal to thread 24424
W20260709 12:17:37.448304 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08001c9900 after lost signal to thread 25570
W20260709 12:17:37.449760 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08002fe620 after lost signal to thread 25662
I20260709 12:17:37.452838 26755 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:37.456183 25659 debug-util.cc:398] Leaking SignalData structure 0x7b0800197460 after lost signal to thread 25737
W20260709 12:17:37.462659 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08002fbc80 after lost signal to thread 25811
W20260709 12:17:37.466732 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08002c77a0 after lost signal to thread 26368
I20260709 12:17:37.497088 26755 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.504024 25709 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.531628 25634 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:37.540935 25636 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:37.555406 25601 consensus_peers.cc:597] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6158b789bb894b2b9ad0c019dde5476f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:37.561446 25785 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:37.562984 26713 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:37.586264 25711 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:37.588008 26713 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:37.606894 25532 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:37.612537 26773 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.613826 25785 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.654696 25785 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:37.687288 25536 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:37.728489 25709 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:37.761111 25750 consensus_peers.cc:597] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab8e959328c44aed94ce17546b6198e2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:37.764523 25709 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:37.766746 26774 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.768452 25630 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:37.772985 26769 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:37.779862 26792 ts_tablet_manager.cc:933] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:37.787667 26776 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.793908 26792 tablet_copy_client.cc:323] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:37.793119 25630 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.795926 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 402bc370028b4bc097d5c2fb4342cdfa from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:37.796653 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 402bc370028b4bc097d5c2fb4342cdfa from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-402bc370028b4bc097d5c2fb4342cdfa
I20260709 12:17:37.804531 25712 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.805868 25723 tablet_copy_source_session.cc:215] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:37.809069 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402bc370028b4bc097d5c2fb4342cdfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:37.830292 26792 tablet_copy_client.cc:806] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:37.831103 26792 tablet_copy_client.cc:670] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:37.835672 26792 tablet_copy_client.cc:538] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:37.866314 26792 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:37.907234 26792 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:37.908253 26792 tablet_bootstrap.cc:492] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:37.908979 26792 ts_tablet_manager.cc:1403] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.043s	user 0.034s	sys 0.008s
I20260709 12:17:37.909724 25536 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:37.912189 26792 raft_consensus.cc:359] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:37.913190 26792 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:37.914064 26792 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:37.995631 26797 ts_tablet_manager.cc:933] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:38.000264 26792 ts_tablet_manager.cc:1434] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.091s	user 0.020s	sys 0.013s
I20260709 12:17:38.014225 26797 tablet_copy_client.cc:323] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:38.015394 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-402bc370028b4bc097d5c2fb4342cdfa received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:38.015923 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-402bc370028b4bc097d5c2fb4342cdfa on tablet 402bc370028b4bc097d5c2fb4342cdfa with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:38.015986 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet a741c75d1e684375b51f415fd0aa0296 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:38.016639 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet a741c75d1e684375b51f415fd0aa0296 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-a741c75d1e684375b51f415fd0aa0296
I20260709 12:17:38.025527 25797 tablet_copy_source_session.cc:215] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:38.029659 26797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a741c75d1e684375b51f415fd0aa0296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:38.049440 26797 tablet_copy_client.cc:806] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:38.050136 26797 tablet_copy_client.cc:670] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:38.053736 26797 tablet_copy_client.cc:538] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:38.062971 26797 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:38.089694 26797 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:38.090634 26797 tablet_bootstrap.cc:492] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:38.091264 26797 ts_tablet_manager.cc:1403] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.009s
I20260709 12:17:38.094168 26797 raft_consensus.cc:359] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:38.095150 26797 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:38.095829 26797 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:38.108566 26797 ts_tablet_manager.cc:1434] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:17:38.109140 26792 ts_tablet_manager.cc:933] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:38.110446 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-a741c75d1e684375b51f415fd0aa0296 received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:38.111050 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-a741c75d1e684375b51f415fd0aa0296 on tablet a741c75d1e684375b51f415fd0aa0296 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:38.118629 26792 tablet_copy_client.cc:323] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:38.120584 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet 6158b789bb894b2b9ad0c019dde5476f from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:38.121235 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet 6158b789bb894b2b9ad0c019dde5476f from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-6158b789bb894b2b9ad0c019dde5476f
I20260709 12:17:38.130590 25648 tablet_copy_source_session.cc:215] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:38.135371 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6158b789bb894b2b9ad0c019dde5476f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:38.155536 26792 tablet_copy_client.cc:806] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:38.158097 26792 tablet_copy_client.cc:670] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:38.164534 26341 raft_consensus.cc:1217] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:38.169497 26792 tablet_copy_client.cc:538] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:38.193946 26792 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:38.210434 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:38.263687 26792 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:38.265055 26792 tablet_bootstrap.cc:492] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:38.270154 26792 ts_tablet_manager.cc:1403] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.076s	user 0.020s	sys 0.014s
I20260709 12:17:38.273818 26792 raft_consensus.cc:359] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:38.274935 26792 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:38.275725 26792 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:38.278874 26792 ts_tablet_manager.cc:1434] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:17:38.286777 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-6158b789bb894b2b9ad0c019dde5476f received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:38.287283 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-6158b789bb894b2b9ad0c019dde5476f on tablet 6158b789bb894b2b9ad0c019dde5476f with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:38.326575 26792 ts_tablet_manager.cc:933] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:38.334316 26792 tablet_copy_client.cc:323] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:38.336396 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet ab8e959328c44aed94ce17546b6198e2 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:38.337003 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet ab8e959328c44aed94ce17546b6198e2 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-ab8e959328c44aed94ce17546b6198e2
I20260709 12:17:38.348003 25797 tablet_copy_source_session.cc:215] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:38.351400 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e959328c44aed94ce17546b6198e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:38.378448 26792 tablet_copy_client.cc:806] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:38.379076 26792 tablet_copy_client.cc:670] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:38.383003 26792 tablet_copy_client.cc:538] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:38.392076 26792 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:38.442555 26792 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:38.444113 26792 tablet_bootstrap.cc:492] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:38.444936 26792 ts_tablet_manager.cc:1403] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.015s
I20260709 12:17:38.448578 26792 raft_consensus.cc:359] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:38.449834 26792 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:38.453362 26792 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:38.459235 25634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:38.459980 25634 raft_consensus.cc:606] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:38.463835 26792 ts_tablet_manager.cc:1434] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:17:38.465975 25797 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-ab8e959328c44aed94ce17546b6198e2 received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:38.466549 25797 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-ab8e959328c44aed94ce17546b6198e2 on tablet ab8e959328c44aed94ce17546b6198e2 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:38.468108 25634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:38.468858 25634 raft_consensus.cc:606] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:38.511986 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0c8da68209b48f8868677e30067d396"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:38.512578 25785 raft_consensus.cc:606] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:38.523561 25634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:38.524209 25634 raft_consensus.cc:606] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:38.580503 25634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:38.581247 25634 raft_consensus.cc:606] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:38.588259 26341 raft_consensus.cc:1217] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:38.601179 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:38.601894 25636 raft_consensus.cc:606] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:38.607378 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:38.609227 26341 raft_consensus.cc:1217] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:38.609556 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:38.610965 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:38.691731 26762 raft_consensus.cc:1064] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:38.695113 26762 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:38.725222 25636 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:38.725270 26339 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:38.727079 25783 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:38.731184 26766 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:38.733304 26766 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:38.735818 26705 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:38.783432 26772 raft_consensus.cc:993] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:38.784116 26772 raft_consensus.cc:1081] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:38.786427 25709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:17:38.787127 25709 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:38.787575 25709 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.812851 26724 raft_consensus.cc:993] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:38.813417 26724 raft_consensus.cc:1081] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:38.815239 26341 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:38.815853 26341 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:38.816226 26341 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.824774 26341 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:38.839841 26773 raft_consensus.cc:993] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:38.840384 26773 raft_consensus.cc:1081] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:38.842043 26337 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:38.844893 25785 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:38.849587 25710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:17:38.850150 25710 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:38.850476 25710 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.839236 26812 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:38.875967 25532 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:38.891768 25710 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:38.810573 25709 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:38.901787 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:38.902472 25637 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:38.902858 25637 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:38.903630 25637 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:38.904807 25637 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.893049 25636 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:38.914502 25637 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:38.915030 25709 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:38.915143 26339 raft_consensus.cc:1217] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:38.916239 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:38.917269 25785 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.917238 25677 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:38.922724 26812 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:38.923280 26812 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:38.924216 26812 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:38.926172 25785 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:38.932885 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:38.933673 25637 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:38.934156 25637 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:38.935191 25637 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:38.937179 25637 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.964632 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:38.965519 25634 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:38.965859 25634 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:38.967099 25634 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:38.968588 25634 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.988777 25634 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:38.991668 26341 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:38.993076 25710 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:38.996045 25677 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:39.000645 25637 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:39.004189 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:17:39.005260 26796 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:39.013477 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:39.014431 25710 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.024930 26796 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:39.037837 25710 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:39.031212 26812 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:39.060039 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:39.060640 25785 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.064138 26796 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:39.058560 26812 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:39.070895 25785 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:39.072300 26812 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:39.071779 26776 raft_consensus.cc:993] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:39.081192 26776 raft_consensus.cc:1081] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:39.117327 26773 raft_consensus.cc:993] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:39.117895 26773 raft_consensus.cc:1081] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:39.129092 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0c8da68209b48f8868677e30067d396"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:39.129621 25636 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:39.129943 25636 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.138159 25636 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:39.142668 25636 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:39.142889 25709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:17:39.143857 26817 raft_consensus.cc:993] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:39.152534 26776 raft_consensus.cc:1064] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:39.152949 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:39.156128 25709 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:39.156446 25710 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.156512 25709 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.160946 26776 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:39.176472 25711 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:39.163295 25709 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:39.164485 26817 raft_consensus.cc:1081] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:39.179445 25709 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:39.180712 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:39.181211 25634 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:39.181488 25634 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:39.182158 25634 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:39.187637 25634 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.166198 26724 raft_consensus.cc:1064] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:39.170353 25710 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:39.193492 25712 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:39.194286 25603 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:39.201622 25634 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:39.202781 25677 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:39.203461 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:39.204027 25782 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.204149 26821 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:39.204653 26821 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:39.205538 26821 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:39.195597 26724 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:39.215536 25534 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.217038 26810 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:39.217636 26810 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:39.218451 25782 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:39.218544 26810 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:39.222142 26341 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:39.220233 26337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:39.228789 26705 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:17:39.227070 26774 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:17:39.237142 26337 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:39.278693 26337 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:39.289877 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 64879.5us
I20260709 12:17:39.296545 25636 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:39.317688 25536 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:39.337880 25712 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:39.362890 26339 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:17:39.364774 26828 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:17:39.333902 26762 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:17:39.380942 26831 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:39.360687 25536 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:39.381930 26337 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:17:39.403792 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45484.2us
W20260709 12:17:39.405025 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 63847.9us
W20260709 12:17:39.451390 26843 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:17:39.491575 26845 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:17:39.501009 25712 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:39.503633 26796 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:39.503901 26337 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:39.504182 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:39.504881 25636 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:39.505214 25636 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:39.505892 25636 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:39.506799 25636 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.507390 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:39.507936 25782 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.508769 25711 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:17:39.509847 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 283022us
I20260709 12:17:39.517977 26341 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:39.520117 26839 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:39.523749 25782 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:39.525454 25636 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:39.530507 26822 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:39.533658 26822 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:39.534644 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:17:39.535782 26796 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:39.543051 25785 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:39.544456 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:39.543529 25534 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:39.534538 26337 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:39.554169 26831 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:39.557905 26837 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:39.562688 25781 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:39.567960 26772 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:39.586125 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 402bc370028b4bc097d5c2fb4342cdfa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:39.595238 25634 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:39.619355 25636 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:39.534821 26822 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:39.628067 25636 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:39.640460 25782 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:39.647665 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.616078 26796 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:39.618348 26762 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:39.687356 26820 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:17:39.649365 26839 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:17:39.695917 26796 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:39.734862 26818 raft_consensus.cc:1064] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:39.744082 26773 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:39.747067 25634 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:39.753821 26848 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:39.756632 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:39.781975 25706 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:39.811029 26822 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.814584 25534 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:39.789717 26772 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:17:39.832927 26836 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:17:39.820237 26818 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:17:39.879551 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 223351us
I20260709 12:17:39.883719 25636 raft_consensus.cc:2955] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:39.898166 26847 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:17:39.826713 26810 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:17:39.933030 25632 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:39.981987 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 57449.3us
I20260709 12:17:39.984995 25536 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:39.988368 25637 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:39.994433 26812 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:40.080103 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 273889us
W20260709 12:17:40.130699 26782 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:17:40.143419 25785 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:40.162468 25783 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:40.160724 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82899.9us
W20260709 12:17:40.206562 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48741.5us
W20260709 12:17:40.212306 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.276921 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 353326us
I20260709 12:17:40.351590 26776 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:40.342474 25782 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:40.424048 26339 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:40.529588 25638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.018s	sys 0.029s
W20260709 12:17:40.529961 25638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.018s	sys 0.029s
I20260709 12:17:40.543782 26835 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:40.559490 25707 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:40.568512 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 430707us
W20260709 12:17:40.594883 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 379863us
W20260709 12:17:40.576352 25632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.022s	sys 0.032s
W20260709 12:17:40.601045 25632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.023s	sys 0.032s
W20260709 12:17:40.671840 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 452805us
W20260709 12:17:40.677613 26862 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:17:40.693864 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69124.4us
I20260709 12:17:40.715794 25709 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:40.751703 25636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.012s	sys 0.028s
W20260709 12:17:40.752465 25636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.012s	sys 0.028s
I20260709 12:17:40.788877 26821 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:17:40.751718 26782 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:17:40.699474 26812 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:17:40.751456 26823 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:17:40.834760 26863 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:17:40.842767 25629 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:40.819375 26857 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:17:40.844425 26864 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:17:40.870980 25786 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:40.789534 26821 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:17:40.887316 26870 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:17:40.894114 26867 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:17:40.881497 26857 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:40.900245 26796 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:40.901505 26857 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:40.903731 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:40.923935 26873 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:17:40.848268 26823 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:17:40.944866 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 74242us
I20260709 12:17:40.945569 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:40.955907 26341 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:40.961621 26821 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:40.929243 26823 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:40.987339 26848 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:40.998368 26769 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:41.003505 25712 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:41.012364 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:41.033070 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:41.033783 25705 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:17:41.039880 25534 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 206) took 1284 ms. Trace:
W20260709 12:17:41.048174 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45151.9us
W20260709 12:17:41.022011 26776 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 12:17:41.040361 25534 rpcz_store.cc:276] 0709 12:17:39.755668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.763697 (+  8029us) service_pool.cc:224] Handling call
0709 12:17:41.039775 (+1276078us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:39.918577 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:39.967558 (+ 48981us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:39.967608 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:17:39.967632 (+    24us) tablet.cc:662] Decoding operations
0709 12:17:39.968032 (+   400us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:39.968092 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:39.968119 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:39.968322 (+   203us) tablet.cc:701] Row locks acquired
0709 12:17:39.968340 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:17:39.968499 (+   159us) write_op.cc:270] Start()
0709 12:17:39.968614 (+   115us) write_op.cc:276] Timestamp: P: 1783599459968452 usec, L: 0
0709 12:17:39.968637 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:17:39.969285 (+   648us) log.cc:844] Serialized 541 byte log entry
0709 12:17:40.132676 (+163391us) op_driver.cc:464] REPLICATION: finished
0709 12:17:40.830691 (+698015us) write_op.cc:301] APPLY: starting
0709 12:17:40.835165 (+  4474us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:40.835269 (+   104us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:40.843067 (+  7798us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:40.878622 (+ 35555us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:40.878853 (+   231us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:40.878902 (+    49us) write_op.cc:312] APPLY: finished
0709 12:17:40.886930 (+  8028us) log.cc:844] Serialized 23 byte log entry
0709 12:17:40.887155 (+   225us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:40.887249 (+    94us) write_op.cc:454] Released schema lock
0709 12:17:40.887294 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:40.887730 (+   436us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:40.887760 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1073,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":657755,"apply.run_cpu_time_us":4726,"apply.run_wall_time_us":67554,"num_ops":1,"prepare.queue_time_us":31857,"prepare.run_cpu_time_us":3087,"prepare.run_wall_time_us":57929,"replication_time_us":163873,"thread_start_us":453956,"threads_started":1}]]}
I20260709 12:17:41.061046 26844 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:41.064522 25752 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:41.065498 26844 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.067199 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:41.058425 26831 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:41.089305 26831 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:41.091988 26831 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:41.067431 26796 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:41.106566 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:41.108619 25603 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:17:41.176620 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 59353.8us
I20260709 12:17:41.114506 26796 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:41.218015 25536 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 111) took 1268 ms. Trace:
I20260709 12:17:41.188364 26878 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:41.214263 25533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 213) took 1080 ms. Trace:
I20260709 12:17:41.164958 26772 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
W20260709 12:17:41.233594 26889 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:17:41.231568 25533 rpcz_store.cc:276] 0709 12:17:40.154008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.154348 (+   340us) service_pool.cc:224] Handling call
0709 12:17:40.374969 (+220621us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5400080610. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286b0b2 00007f29bb7e4759 00007f29bb77ca76 00007f29bb74dd58 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc
0709 12:17:40.603453 (+228484us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:17:41.214175 (+610722us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6573,"reported_tablets":1,"spinlock_wait_cycles":540961408}
I20260709 12:17:41.240391 25634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:41.241053 25634 raft_consensus.cc:606] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:41.230667 25536 rpcz_store.cc:276] 0709 12:17:39.949851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.950043 (+   192us) service_pool.cc:224] Handling call
0709 12:17:41.217930 (+1267887us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:40.029018 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:40.220043 (+191025us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:40.220082 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:17:40.220102 (+    20us) tablet.cc:662] Decoding operations
0709 12:17:40.220587 (+   485us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:40.220655 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:40.220688 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:40.222287 (+  1599us) tablet.cc:701] Row locks acquired
0709 12:17:40.222301 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:17:40.227912 (+  5611us) write_op.cc:270] Start()
0709 12:17:40.230513 (+  2601us) write_op.cc:276] Timestamp: P: 1783599460226653 usec, L: 0
0709 12:17:40.230544 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:17:40.671006 (+440462us) log.cc:844] Serialized 536 byte log entry
0709 12:17:40.899788 (+228782us) op_driver.cc:464] REPLICATION: finished
0709 12:17:40.917239 (+ 17451us) write_op.cc:301] APPLY: starting
0709 12:17:40.920901 (+  3662us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:40.939660 (+ 18759us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:40.947803 (+  8143us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598ae28 00007f29b93c5581 00007f29b93c4d58 00007f29b93c99f1 00007f29b93c99a9 00007f29b93c9939 00007f29b93c9901 00007f29b93c8b2d 0000000000388634 0000000000388479
0709 12:17:40.943043 (+ -4760us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:40.956732 (+ 13689us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:40.975579 (+ 18847us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:40.982015 (+  6436us) write_op.cc:312] APPLY: finished
0709 12:17:41.010587 (+ 28572us) log.cc:844] Serialized 23 byte log entry
0709 12:17:41.012689 (+  2102us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:41.012774 (+    85us) write_op.cc:454] Released schema lock
0709 12:17:41.012812 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:41.073950 (+ 61138us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b5400001c80. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b92f9161 00007f29b93ce33d 00007f29b93c73c2 00007f29b93c6b53 00007f29b93cad61 00007f29b93cad19 00007f29b93caca9 00007f29b93cac71 00007f29b93c9e9d 0000000000388634 0000000000388479
0709 12:17:41.074193 (+   243us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:41.074223 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":201311,"reported_tablets":1,"spinlock_wait_cycles":13056,"child_traces":[["op",{"apply.queue_time_us":10845,"apply.run_cpu_time_us":4655,"apply.run_wall_time_us":160993,"num_ops":1,"prepare.queue_time_us":18469,"prepare.run_cpu_time_us":13193,"prepare.run_wall_time_us":922644,"replication_time_us":669080,"spinlock_wait_cycles":845266560}]]}
I20260709 12:17:41.231091 26878 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:17:41.253644 26888 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:17:41.255095 25634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:41.255774 25634 raft_consensus.cc:606] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
W20260709 12:17:41.259308 26890 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:17:41.263806 26796 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:41.265360 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:41.234578 26774 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:41.255929 26878 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:41.282295 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:41.286388 25784 raft_consensus.cc:606] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
W20260709 12:17:41.291877 26893 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:17:41.284339 26822 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:41.292574 26822 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:41.310864 26822 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:41.296515 26774 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:41.362844 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 188203us
I20260709 12:17:41.398386 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:41.403936 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 211403us
W20260709 12:17:41.429102 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 323842us
I20260709 12:17:41.428390 26774 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:17:41.431542 26895 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 12:17:41.456017 26863 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.283s	user 0.004s	sys 0.001s
I20260709 12:17:41.457940 26878 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:41.460382 26878 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:41.468860 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:41.475019 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76837.7us
I20260709 12:17:41.479000 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:41.452144 26831 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:41.500207 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:41.472806 26769 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:41.499148 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:41.509893 26769 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:17:41.513027 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48542.4us
I20260709 12:17:41.477978 26903 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:41.524180 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:41.524386 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:41.525667 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:41.510632 26848 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:41.527220 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:41.535461 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:41.543128 25785 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:41.503917 26831 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:41.528043 26796 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.609866 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:17:41.614964 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 81695us
W20260709 12:17:41.622056 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 121932us
I20260709 12:17:41.611387 26831 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:17:41.630833 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 107946us
W20260709 12:17:41.623196 26867 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:17:41.632606 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:41.618893 26796 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:41.598577 26848 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:41.622741 26769 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:41.528617 26903 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:41.637511 26341 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:41.641911 26796 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:41.668555 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:41.678205 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b06b96578363435cbde83279e86e5c86"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:41.682997 25636 raft_consensus.cc:606] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:41.681499 26817 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:17:41.722733 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 89555.4us
I20260709 12:17:41.712824 26848 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:41.739591 25711 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:41.713091 26817 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:41.657407 26913 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:41.939565 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55926.9us
I20260709 12:17:41.950994 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:17:41.788204 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 146971us
W20260709 12:17:41.939688 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 93211.8us
I20260709 12:17:41.863081 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:41.833721 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:41.778648 26796 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:41.861768 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:41.831617 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:41.785620 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:41.748045 26774 raft_consensus.cc:993] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:41.976601 25630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:41.977311 25630 raft_consensus.cc:606] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
W20260709 12:17:41.962059 26858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.007s	sys 0.016s
I20260709 12:17:41.742726 26903 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:17:41.990882 26858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.007s	sys 0.017s
I20260709 12:17:42.011053 26776 raft_consensus.cc:1081] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:42.148941 25532 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 211) took 1558 ms. Trace:
I20260709 12:17:42.149555 25532 rpcz_store.cc:276] 0709 12:17:40.590942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.591473 (+   531us) service_pool.cc:224] Handling call
0709 12:17:40.897948 (+306475us) spinlock_profiling.cc:243] Waited 94.7 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:17:42.148844 (+1250896us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31289,"reported_tablets":1,"spinlock_wait_cycles":217892224}
I20260709 12:17:41.995335 26344 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:42.174542 26848 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:42.240651 25630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:42.239792 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:17:42.165135 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 57919.3us
W20260709 12:17:42.260371 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 289756us
I20260709 12:17:42.259055 25630 raft_consensus.cc:606] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.246683 25536 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:42.268424 25536 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:42.270653 25601 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:17:42.297461 26309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.022s	sys 0.031s
W20260709 12:17:42.298185 26309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.023s	sys 0.031s
I20260709 12:17:42.275394 26906 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.275404 26817 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:42.274442 26910 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:42.264765 26848 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:17:42.275552 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 315228us
W20260709 12:17:42.314625 26341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.028s	sys 0.046s
I20260709 12:17:42.307502 26904 raft_consensus.cc:993] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
W20260709 12:17:42.315201 26341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.028s	sys 0.046s
I20260709 12:17:42.314345 26817 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:42.357106 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 49554.8us
W20260709 12:17:42.359218 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 50299.2us
I20260709 12:17:42.352749 26904 raft_consensus.cc:1081] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:42.384023 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.317588 26910 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:17:42.386849 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 136946us
W20260709 12:17:42.391563 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 77061.6us
W20260709 12:17:42.392472 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 121895us
I20260709 12:17:42.397348 26307 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:42.398361 26911 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:42.398615 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 73463.5us
W20260709 12:17:42.400782 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78319.7us
I20260709 12:17:42.401538 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.405710 26820 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.398618 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:42.409307 26906 raft_consensus.cc:993] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:42.409837 26906 raft_consensus.cc:1081] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:42.409962 26911 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.418644 25711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:17:42.419344 25711 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:42.419741 25711 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.433657 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:42.450911 26871 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.451509 26871 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.454798 25713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
W20260709 12:17:42.456290 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54822.6us
I20260709 12:17:42.468041 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.469638 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.476022 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.478600 26871 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.475699 26916 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.465286 25713 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:42.490172 25713 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.463581 26904 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.490841 26904 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.492980 26904 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.487596 26871 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.493705 26871 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:42.488176 26928 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:17:42.496162 26916 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.466797 26773 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.496965 26773 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.498509 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:42.489161 26906 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.489670 26910 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:42.494308 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.499842 25711 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.502925 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:42.515686 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.513652 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.521903 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:42.522341 26929 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.522552 25631 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:42.499118 26773 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.530059 25711 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.514618 26821 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.526096 26817 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.516630 26914 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:42.541672 26817 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.502323 26906 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.543414 26821 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.545883 26837 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.547636 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.553555 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.524385 25631 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:42.554610 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.546471 26837 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.557780 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:42.555210 25631 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.562618 26929 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.563148 26929 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.558758 26906 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:42.563714 26902 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.542241 25713 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.542261 26914 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.565505 26772 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.588995 26837 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.607137 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.607986 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.596045 26772 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.614032 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.614773 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:42.615203 26934 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.615617 25783 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.617455 25713 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.604105 26848 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:42.622391 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:42.623168 25630 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:42.623519 25630 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:42.586618 26902 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.625183 26930 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.629848 26821 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.630985 25630 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.614384 26865 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:42.632314 25630 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.632571 26865 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.646350 26772 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:42.635872 26902 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.649246 26817 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.652143 25783 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:42.658560 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:42.659196 25784 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.662810 25630 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:42.670027 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:42.671295 25635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:42.671844 25635 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:42.672197 25635 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.676326 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.677457 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.678068 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.679050 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.666963 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:42.682399 26817 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.684145 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.684511 26817 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.685333 25711 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:42.682416 26837 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:17:42.665829 26942 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:17:42.687000 26837 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.690315 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.689646 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:42.692003 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:42.690233 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:42.693410 26865 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.697917 25784 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:42.683238 26929 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.709414 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.712334 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:42.714630 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.679955 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:42.692112 26817 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.703500 25635 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.716518 25638 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:42.704299 26836 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:42.723340 26836 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:42.731918 26840 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:42.733098 26837 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:42.733717 25785 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:42.736655 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.737030 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:42.738875 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.740677 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.741063 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.742805 26906 raft_consensus.cc:993] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:42.743614 25637 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:17:42.744007 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.724299 26836 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.738385 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.750856 25707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:42.747221 26840 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:42.751549 25707 raft_consensus.cc:606] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:42.759971 26923 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:42.760748 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:42.762045 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.760588 26923 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.764089 25631 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.757297 26914 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.767679 25635 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:42.744042 26906 raft_consensus.cc:1081] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:42.773876 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:42.777197 25711 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.780617 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.791203 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:17:42.793356 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:42.794346 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:42.793099 26871 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.795571 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:42.796232 26336 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.796343 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:42.797690 26850 raft_consensus.cc:993] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:42.798806 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.810971 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.813047 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:17:42.814949 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.814318 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:17:42.817111 26910 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.818708 26910 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:42.798188 26850 raft_consensus.cc:1081] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:42.792090 26773 raft_consensus.cc:993] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:42.797950 26840 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.819391 26844 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.821396 26913 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:42.820328 26820 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.842453 25635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51522
I20260709 12:17:42.844012 25635 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:42.844370 25635 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.844750 26868 raft_consensus.cc:1081] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:42.825124 25631 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:42.827658 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:42.829016 25711 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:42.841926 26831 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:42.852126 26914 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.855850 25635 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.859577 26342 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:42.860144 26342 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:42.842772 26913 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:42.860585 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b06b96578363435cbde83279e86e5c86"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:42.861140 26344 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:42.861529 26344 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.861353 26913 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.863392 26913 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.860484 26342 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.865677 26820 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:42.868233 26820 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:42.870110 25635 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.869169 26820 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.888741 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:42.871438 26947 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:42.872154 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:42.889846 25713 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:42.890161 25713 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:42.889246 26820 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.890853 25713 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.893673 25713 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.874260 26336 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:42.883490 26923 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:42.895924 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.897293 26342 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.901103 26342 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.902011 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.903057 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.900468 26947 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:42.908149 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:42.908685 25781 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.910351 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.911605 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:42.913342 25752 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.916036 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:42.916952 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:42.904265 26906 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.909389 26835 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.903735 26902 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:42.935444 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:17:42.931341 26937 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.949045 26949 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.953871 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.964517 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.949853 25781 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
W20260709 12:17:42.951233 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.967748 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:42.967857 25713 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:17:42.969828 26947 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:42.970788 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:42.975399 26871 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.987970 26344 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:42.989996 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:42.995041 25632 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:42.997957 25786 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:43.001237 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:43.004647 26837 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.011608 25601 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:43.011981 26947 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.007886 25632 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:43.018214 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:43.012013 26914 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.022001 25601 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:43.014389 26871 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.024387 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:43.024748 26914 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:43.029470 26344 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:43.023888 26835 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.030635 26773 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.031359 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:43.031994 25631 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:43.022843 26837 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.032327 25631 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:43.033195 25631 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.034310 25631 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.026814 25603 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:43.029642 26914 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.042857 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:43.043416 25713 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.045156 26947 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:43.047854 25632 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.048996 25632 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.058566 26947 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:43.059504 26947 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:43.065974 25536 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 112) took 1560 ms. Trace:
I20260709 12:17:43.067217 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:17:43.089020 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:17:43.070883 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 402bc370028b4bc097d5c2fb4342cdfa with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:43.091974 26956 tablet_replica.cc:333] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:43.093761 26956 raft_consensus.cc:2243] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.066395 25536 rpcz_store.cc:276] 0709 12:17:41.505253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.505421 (+   168us) service_pool.cc:224] Handling call
0709 12:17:43.065915 (+1560494us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:42.425186 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:42.429964 (+  4778us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:42.430010 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:17:42.430029 (+    19us) tablet.cc:662] Decoding operations
0709 12:17:42.430519 (+   490us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:42.430606 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:42.430655 (+    49us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:17:42.431038 (+   383us) tablet.cc:701] Row locks acquired
0709 12:17:42.431050 (+    12us) write_op.cc:260] PREPARE: finished
0709 12:17:42.431198 (+   148us) write_op.cc:270] Start()
0709 12:17:42.431324 (+   126us) write_op.cc:276] Timestamp: P: 1783599462431152 usec, L: 0
0709 12:17:42.431354 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:17:42.432072 (+   718us) log.cc:844] Serialized 982 byte log entry
0709 12:17:42.766954 (+334882us) spinlock_profiling.cc:243] Waited 4.5 ms on lock 0x7b18000a9900. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:17:42.849123 (+ 82169us) op_driver.cc:464] REPLICATION: finished
0709 12:17:43.055629 (+206506us) write_op.cc:301] APPLY: starting
0709 12:17:43.055717 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:43.055844 (+   127us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:43.055868 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:43.056234 (+   366us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:43.056416 (+   182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:17:43.056469 (+    53us) write_op.cc:312] APPLY: finished
0709 12:17:43.056671 (+   202us) log.cc:844] Serialized 29 byte log entry
0709 12:17:43.057433 (+   762us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:43.057527 (+    94us) write_op.cc:454] Released schema lock
0709 12:17:43.057575 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:43.057890 (+   315us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:43.057919 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":639592,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":195930,"apply.run_cpu_time_us":1668,"apply.run_wall_time_us":2268,"num_ops":2,"prepare.queue_time_us":99174,"prepare.run_cpu_time_us":8464,"prepare.run_wall_time_us":84441,"raft.queue_time_us":71376,"raft.run_cpu_time_us":22356,"raft.run_wall_time_us":156833,"replication_time_us":417599,"spinlock_wait_cycles":10360064,"thread_start_us":320054,"threads_started":4,"wal-append.queue_time_us":47150}]]}
I20260709 12:17:43.094919 26956 raft_consensus.cc:2272] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.095191 25632 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:43.085887 25532 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.098416 26956 ts_tablet_manager.cc:1916] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:43.087581 25631 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:43.103375 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:17:43.104276 26914 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.104745 26914 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:43.105648 26914 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.109488 25713 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:43.111590 26837 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.112442 26871 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.113812 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:17:43.114646 26871 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.116478 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:43.117257 26871 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.117990 26871 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:43.119092 26871 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.119867 26956 ts_tablet_manager.cc:1929] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:43.120445 26956 log.cc:1199] T 402bc370028b4bc097d5c2fb4342cdfa P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/402bc370028b4bc097d5c2fb4342cdfa
I20260709 12:17:43.122550 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 402bc370028b4bc097d5c2fb4342cdfa (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:43.124822 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:43.144155 25536 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:43.147125 25536 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:43.150118 25520 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a741c75d1e684375b51f415fd0aa0296 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 955233c139344131a847044f17f8d348 (127.23.217.67:41061) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 955233c139344131a847044f17f8d348 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.152803 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:43.153451 25781 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.168916 25534 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 207) took 2009 ms. Trace:
I20260709 12:17:43.169404 25534 rpcz_store.cc:276] 0709 12:17:41.159529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.160824 (+  1295us) service_pool.cc:224] Handling call
0709 12:17:41.647057 (+486233us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:17:43.168842 (+1521785us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":585251,"reported_tablets":6,"spinlock_wait_cycles":2511104}
I20260709 12:17:43.170023 25635 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.177774 25631 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.190872 26840 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.181905 25781 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:17:43.187351 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:43.187090 25532 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:43.228570 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:43.231252 26837 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.296783 26339 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:43.300660 26902 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:43.324142 25713 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:43.310158 25532 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:43.328415 25532 catalog_manager.cc:5697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:43.334275 26837 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.341140 26871 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:43.348557 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6158b789bb894b2b9ad0c019dde5476f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:43.353376 25783 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.369822 25709 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.373428 26884 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:43.373963 26339 raft_consensus.cc:2955] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.405696 25631 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.417986 25532 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:43.421082 25532 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:43.426092 26933 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.437491 25782 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:43.437865 26964 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:17:43.424330 25532 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 2, term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65), 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from VOTER to NON_VOTER, now has a pending config: VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65), VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66), VOTER 955233c139344131a847044f17f8d348 (127.23.217.67), VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.465958 25786 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.479660 25709 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:43.484329 26930 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:43.468236 26914 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:17:43.474953 26950 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:17:43.525625 26914 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:43.526229 26914 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.528760 26914 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:43.589970 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 6158b789bb894b2b9ad0c019dde5476f with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:43.593925 26956 tablet_replica.cc:333] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:43.596629 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:43.598825 25635 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:43.603348 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:43.616657 25636 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:43.627166 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:43.628219 25786 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:43.634543 26848 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:43.641789 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48405.2us
W20260709 12:17:43.653401 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57518.5us
I20260709 12:17:43.654673 25705 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:43.658638 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:43.660764 26848 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.670501 25705 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:43.676183 26336 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:43.676927 26930 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:43.681182 26914 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:43.680778 26837 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:17:43.703447 25636 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:43.711783 25786 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:43.725173 26968 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:43.747879 26848 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.795502 26871 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:43.829852 25782 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:43.833123 26773 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.856631 26930 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:43.857868 25534 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65), VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66), VOTER 955233c139344131a847044f17f8d348 (127.23.217.67), VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.963716 26956 raft_consensus.cc:2243] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.965173 26956 raft_consensus.cc:2272] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.956326 25635 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:43.970772 26956 ts_tablet_manager.cc:1916] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:43.985628 26339 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:43.987603 26773 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:43.997388 26902 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:43.999344 25783 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:44.002007 26837 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:44.006740 25783 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:44.052502 26339 raft_consensus.cc:2955] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:44.074923 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a741c75d1e684375b51f415fd0aa0296 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20260709 12:17:44.088209 26956 ts_tablet_manager.cc:1929] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:44.089100 26956 log.cc:1199] T 6158b789bb894b2b9ad0c019dde5476f P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/6158b789bb894b2b9ad0c019dde5476f
I20260709 12:17:44.101259 25536 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 5, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:44.120469 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet a741c75d1e684375b51f415fd0aa0296 with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:44.122296 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 6158b789bb894b2b9ad0c019dde5476f (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:44.147017 26982 tablet_replica.cc:333] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:44.149266 26982 raft_consensus.cc:2243] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.150480 26982 raft_consensus.cc:2272] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.160236 26982 ts_tablet_manager.cc:1916] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:44.215420 26982 ts_tablet_manager.cc:1929] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:17:44.216006 26982 log.cc:1199] T a741c75d1e684375b51f415fd0aa0296 P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/a741c75d1e684375b51f415fd0aa0296
I20260709 12:17:44.218539 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet a741c75d1e684375b51f415fd0aa0296 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:44.257983 26835 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.258715 26835 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:44.261176 26835 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:44.261977 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:44.266353 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:44.267201 25783 raft_consensus.cc:2410] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:44.268913 25601 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:44.269722 26837 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:44.365211 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:44.452934 26933 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:44.453864 26933 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:44.457175 26933 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:44.473115 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:17:44.474452 25632 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:44.478404 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:17:44.479130 25783 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:44.497128 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:17:44.497959 26336 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:44.510921 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:44.511854 26933 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:44.512233 26933 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:44.512661 26933 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.529374 26933 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:44.554093 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:44.554930 25632 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:17:44.556285 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:44.557066 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:44.560314 26933 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:44.573374 25631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:44.574224 25631 raft_consensus.cc:606] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:44.579274 25631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:44.579947 25631 raft_consensus.cc:606] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:44.582986 25783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:44.585355 25786 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:44.585938 25783 raft_consensus.cc:606] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:44.586436 25783 raft_consensus.cc:614] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:44.592501 25783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:44.593137 25783 raft_consensus.cc:606] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:44.595830 25783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:44.597265 25783 raft_consensus.cc:606] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:44.598593 26336 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:44.600132 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:44.600190 25707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:44.601089 25707 raft_consensus.cc:606] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:44.601830 26933 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:44.602684 26933 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:44.603120 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:44.604313 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:44.603730 26933 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:44.605434 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:44.645361 25536 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:44.875643 26852 raft_consensus.cc:993] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:44.876173 26852 raft_consensus.cc:1081] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:44.878036 25631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:44.878773 25631 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:44.879191 25631 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.886713 25631 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:44.889289 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:44.889889 25705 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.892551 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:44.893301 25783 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:44.893744 25783 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:44.894591 25783 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:44.896018 25783 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.899849 25631 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:44.908993 25783 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:44.913061 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:44.915740 26837 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:44.916236 26837 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:44.918115 26837 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:44.924506 25705 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:44.941552 25536 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:44.956416 26837 raft_consensus.cc:993] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:44.956946 26837 raft_consensus.cc:1081] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:44.958760 26336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:44.959390 26336 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:44.959806 26336 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.967674 26336 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:44.971513 26336 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:44.972683 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:44.974103 25783 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.978229 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:44.980907 25632 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:44.981292 25632 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:44.992215 25632 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:44.993623 25632 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.998410 25783 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:17:45.000031 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:45.000993 26968 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:45.013438 25632 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:17:45.019243 26968 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:45.020431 26968 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:45.034170 25536 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 3 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.064680 26933 raft_consensus.cc:993] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:45.065237 26933 raft_consensus.cc:1081] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:45.067273 25631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51522
I20260709 12:17:45.067919 25631 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:45.068367 25631 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:45.076857 26336 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:45.080058 26983 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:45.103510 25783 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:45.116837 26884 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:45.117502 26837 raft_consensus.cc:993] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:45.118129 26837 raft_consensus.cc:1081] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:45.136807 26339 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:45.137530 26339 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:45.137980 26339 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:45.144858 25632 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:45.145306 26339 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:45.148195 26920 raft_consensus.cc:993] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:45.148916 26920 raft_consensus.cc:1081] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:45.148715 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:45.149708 25632 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:45.150063 25632 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:45.151065 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56168
I20260709 12:17:45.151656 26344 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:45.151996 26344 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:45.148255 26983 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:17:45.159166 26344 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:45.161876 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:45.162397 25635 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:45.167176 25631 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:45.171347 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:45.171936 25705 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:45.172240 25705 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:45.172955 25705 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:45.174082 25705 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:45.179769 25635 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:45.183663 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:17:45.184746 26968 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:45.185242 26968 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:45.185485 25629 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:45.186168 26968 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:17:45.194797 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 79678.1us
I20260709 12:17:45.198973 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:45.199548 25783 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:45.199709 25705 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:17:45.199914 25783 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:45.200743 26344 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:45.200991 25783 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:45.231237 25783 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:45.151057 25632 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:45.204013 25631 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:45.205437 25603 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:45.224076 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:45.238174 25784 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:45.239004 26773 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:45.239586 26773 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:45.240386 26773 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:45.241524 25783 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:45.245622 25784 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:17:45.253603 25632 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:45.277606 26339 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:45.293563 25536 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.318101 25533 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:45.335603 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54632.1us
I20260709 12:17:45.380312 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:45.381079 25705 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:45.389015 25705 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:45.376916 25632 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:45.394680 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:17:45.395706 26989 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:45.446920 25632 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:17:45.493469 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55007.2us
I20260709 12:17:45.442003 26989 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:45.500391 26989 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:45.516460 26968 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:17:45.535595 25711 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:45.536429 25784 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:45.537364 27001 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:45.538091 26968 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:45.551913 25705 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:45.560105 25783 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:45.562677 27005 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:45.569806 25783 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:45.569972 26344 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:45.574433 26991 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:45.606779 25533 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.635670 25786 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:45.587431 27008 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:17:45.650090 27000 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:17:45.672001 25781 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:45.678980 27001 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:45.678757 27008 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:45.690285 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:45.698644 26344 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:45.708148 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab8e959328c44aed94ce17546b6198e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:17:45.718237 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 57024.2us
W20260709 12:17:45.719857 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 72788.1us
I20260709 12:17:45.760823 25533 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 3 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.782398 25632 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:45.805999 26989 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:45.813401 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet ab8e959328c44aed94ce17546b6198e2 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:45.836400 27017 tablet_replica.cc:333] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:45.843271 27017 raft_consensus.cc:2243] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:45.848564 27017 raft_consensus.cc:2272] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:45.866077 27017 ts_tablet_manager.cc:1916] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:45.867854 25707 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:45.871263 27005 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:45.972057 25707 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:45.972455 27017 ts_tablet_manager.cc:1929] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:17:45.973002 27017 log.cc:1199] T ab8e959328c44aed94ce17546b6198e2 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/ab8e959328c44aed94ce17546b6198e2
I20260709 12:17:45.974074 26989 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:45.993012 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet ab8e959328c44aed94ce17546b6198e2 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:46.024466 25631 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:46.027206 27015 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:17:46.027443 26989 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:46.011719 27016 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:17:46.100876 25786 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a760df354a14284b004635d45d88a3e"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:46.101481 25786 raft_consensus.cc:606] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:46.116335 25631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:46.117264 25631 raft_consensus.cc:606] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:46.126632 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:46.127314 25709 raft_consensus.cc:606] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:46.134883 25631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e13025a828f141378027526ae4f0b002"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:46.135646 25631 raft_consensus.cc:606] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:46.138906 25707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:46.139428 25707 raft_consensus.cc:606] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:46.140551 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:46.142232 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:46.141242 25783 raft_consensus.cc:2955] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:46.143081 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:46.253281 27005 raft_consensus.cc:993] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:46.253819 27020 raft_consensus.cc:1081] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:46.255228 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e13025a828f141378027526ae4f0b002"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:46.255764 26344 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.256078 26344 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.262591 26344 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.265221 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:46.265980 25632 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.266304 25632 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:46.266721 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:46.267443 26344 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:46.267306 25632 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.269167 25632 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.267369 25783 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.289366 25632 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:46.291271 25783 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:46.291244 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:17:46.292563 26989 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.293148 26989 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:46.294236 26989 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.312089 27006 raft_consensus.cc:993] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:46.312634 27008 raft_consensus.cc:1081] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:46.314160 25783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:46.314662 25783 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.315117 25783 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.318636 25534 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:46.323463 25783 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.333202 25783 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:46.333518 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:46.334421 25707 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.333829 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:46.334751 25707 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:46.335016 26344 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.335667 25707 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.337114 25707 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.344432 26344 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:46.346222 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:46.347600 27014 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.348057 27014 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:46.348913 27014 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.375358 25707 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:46.380559 25534 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.396821 27006 raft_consensus.cc:993] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:46.397446 27008 raft_consensus.cc:1081] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:46.398948 25632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51522
I20260709 12:17:46.399380 25632 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.399670 25632 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.404073 25632 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.405876 25632 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:46.406580 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:46.407347 26344 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.408790 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:46.409435 25707 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.409788 25707 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:46.410667 25707 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.411988 25707 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.419672 25707 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:46.420943 26344 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:46.421005 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:46.422735 27020 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.423223 27020 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:46.424015 27020 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.436745 25534 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.522073 27014 raft_consensus.cc:993] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:46.522621 26920 raft_consensus.cc:1081] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:46.524350 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a760df354a14284b004635d45d88a3e"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56168
I20260709 12:17:46.525075 26344 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.525522 26344 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.536494 26344 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.539359 26344 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:46.539381 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:46.539824 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:46.540187 25707 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.540374 25783 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.540683 25783 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:46.543931 25783 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.545554 25783 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.546660 25707 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:46.547883 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:46.548729 26968 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.549183 26968 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:46.550058 26968 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.554257 25783 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:17:46.575392 27005 raft_consensus.cc:993] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:46.576120 27005 raft_consensus.cc:1081] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:46.582445 25534 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:46.583817 25783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50982
I20260709 12:17:46.584440 25783 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.584741 25783 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.599581 25783 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.602869 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:46.603538 26339 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.607172 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:46.608109 25631 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.608541 25631 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:46.609338 26339 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:46.602528 25783 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:46.609630 25631 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.610723 25750 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:46.611678 25631 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.611801 27014 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.614089 27014 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:46.615001 27014 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:46.633502 25534 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.635376 25631 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:46.791553 25709 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:46.793267 27022 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:46.854664 26339 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:46.862582 27035 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:46.877462 25783 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:46.882719 26968 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:46.886817 25629 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:46.888558 26968 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:17:46.921233 25709 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:46.940112 25711 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:46.941731 27001 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:17:46.977324 25783 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:46.977489 25676 consensus_peers.cc:597] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 571c5e0767064f3e86479b794294d327. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:46.980788 25707 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:46.982867 27006 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:46.987396 25632 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:46.996865 26968 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:17:47.027606 27045 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:47.047564 26339 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:47.049659 27001 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 12:17:47.072400 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 67527.6us
W20260709 12:17:47.073385 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 72832us
I20260709 12:17:47.078075 25632 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:47.083829 27036 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:17:47.114616 25785 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:47.116711 26989 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:47.155689 25786 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.151202 27047 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.194577 25631 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.204909 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:47.206390 25711 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:47.224102 26339 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:47.226109 27052 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:47.239845 25786 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:47.239980 25629 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:47.241788 27047 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:47.259302 25532 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:17:47.275630 25676 consensus_peers.cc:597] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 31ccbf41d24e4275a7f921db24b940d4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:47.287772 27045 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:47.320556 27008 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.322525 25629 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.379640 25786 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.384900 25783 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:47.401232 25705 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:47.401547 25750 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3c8e03aa6454e118cd3f9f25c785bef. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:47.403321 27035 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:47.406457 25632 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:47.409559 27022 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:47.408890 25535 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.425187 25705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:47.425992 25705 raft_consensus.cc:606] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:47.425379 27052 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.429673 25783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b74717dca3554e889066a717cfd1527a"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:47.430437 25783 raft_consensus.cc:606] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:47.433414 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:47.437148 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:47.438114 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:47.451567 27068 ts_tablet_manager.cc:933] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:47.464306 25629 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.498834 25711 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.503767 25786 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:47.515148 27068 tablet_copy_client.cc:323] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:47.514518 25532 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.517305 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 571c5e0767064f3e86479b794294d327 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:47.517886 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 571c5e0767064f3e86479b794294d327 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-571c5e0767064f3e86479b794294d327
I20260709 12:17:47.526000 25705 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:47.526139 25750 consensus_peers.cc:597] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b907e0d1516b4c78a2a977e29402bff9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:47.530928 25723 tablet_copy_source_session.cc:215] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:47.535127 27068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571c5e0767064f3e86479b794294d327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:47.534667 27049 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:47.539829 25629 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:47.541847 27052 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:47.542333 27008 raft_consensus.cc:993] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:47.542996 27008 raft_consensus.cc:1081] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:47.546659 25786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:47.547322 25786 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:47.547735 25786 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:47.554939 27068 tablet_copy_client.cc:806] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:47.555687 27068 tablet_copy_client.cc:670] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:47.562476 25786 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:47.568449 25786 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:47.570133 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:47.571004 26339 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:47.573114 27068 tablet_copy_client.cc:538] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:47.570221 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:47.575379 25711 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:47.575726 25711 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:47.576691 25711 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:47.587020 27068 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:47.589787 27052 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.578217 25711 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:47.595289 25709 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.595883 25632 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.603358 26339 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:47.604866 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:47.605822 27049 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:47.606357 27049 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:47.607309 27049 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:47.620091 25535 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:47.638712 25711 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:47.699437 26920 raft_consensus.cc:993] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:47.700038 26920 raft_consensus.cc:1081] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:47.702068 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b74717dca3554e889066a717cfd1527a"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:47.702684 25636 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:47.703127 25636 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:47.705878 25534 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.711373 25636 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:47.720672 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:47.721319 25711 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:47.724362 25636 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:47.724267 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:47.725293 25783 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:47.725648 25783 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:47.726588 25783 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:47.728241 25783 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:47.729574 25711 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:47.730880 25603 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:47.731783 27057 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:47.732975 27057 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:47.733850 27057 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:47.737227 25783 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:47.745865 27068 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:47.746922 27068 tablet_bootstrap.cc:492] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:47.747697 27068 ts_tablet_manager.cc:1403] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.161s	user 0.051s	sys 0.031s
I20260709 12:17:47.751391 27068 raft_consensus.cc:359] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:47.752532 27068 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:47.753228 27068 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:47.756137 27068 ts_tablet_manager.cc:1434] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:17:47.761606 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-571c5e0767064f3e86479b794294d327 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:47.762116 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-571c5e0767064f3e86479b794294d327 on tablet 571c5e0767064f3e86479b794294d327 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:47.763975 25532 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.816561 27068 ts_tablet_manager.cc:933] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:47.818518 27068 tablet_copy_client.cc:323] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:47.820492 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 31ccbf41d24e4275a7f921db24b940d4 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:47.821062 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 31ccbf41d24e4275a7f921db24b940d4 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-31ccbf41d24e4275a7f921db24b940d4
I20260709 12:17:47.835211 25723 tablet_copy_source_session.cc:215] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:47.838325 27068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ccbf41d24e4275a7f921db24b940d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:47.860984 27068 tablet_copy_client.cc:806] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:47.861737 27068 tablet_copy_client.cc:670] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:47.888623 27068 tablet_copy_client.cc:538] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:47.897588 27068 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:47.943490 26339 raft_consensus.cc:1217] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:47.952811 27068 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:47.953933 27068 tablet_bootstrap.cc:492] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:47.955165 27068 ts_tablet_manager.cc:1403] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.058s	user 0.027s	sys 0.012s
I20260709 12:17:47.958349 27068 raft_consensus.cc:359] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:47.959215 27068 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:47.959983 27068 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:47.962527 27068 ts_tablet_manager.cc:1434] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:17:47.979663 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-31ccbf41d24e4275a7f921db24b940d4 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:47.980170 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-31ccbf41d24e4275a7f921db24b940d4 on tablet 31ccbf41d24e4275a7f921db24b940d4 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:48.017314 27068 ts_tablet_manager.cc:933] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:48.026036 25711 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:48.028792 26920 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:48.036410 27068 tablet_copy_client.cc:323] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:48.044104 25797 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet c3c8e03aa6454e118cd3f9f25c785bef from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:48.044760 25797 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet c3c8e03aa6454e118cd3f9f25c785bef from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-c3c8e03aa6454e118cd3f9f25c785bef
I20260709 12:17:48.082787 26344 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:48.084455 26920 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:48.095811 27082 ts_tablet_manager.cc:933] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:48.107225 27082 tablet_copy_client.cc:323] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:48.109066 25796 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet b907e0d1516b4c78a2a977e29402bff9 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:48.109699 25796 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet b907e0d1516b4c78a2a977e29402bff9 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-b907e0d1516b4c78a2a977e29402bff9
I20260709 12:17:48.112442 25797 tablet_copy_source_session.cc:215] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:48.124725 25796 tablet_copy_source_session.cc:215] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:48.128547 27082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b907e0d1516b4c78a2a977e29402bff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:48.147202 25709 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:48.149161 27059 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:48.187317 27082 tablet_copy_client.cc:806] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:48.189314 27082 tablet_copy_client.cc:670] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:48.194128 27082 tablet_copy_client.cc:538] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:48.219939 27082 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:48.315032 27068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c8e03aa6454e118cd3f9f25c785bef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:48.324110 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 66136.4us
I20260709 12:17:48.326506 25783 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:48.372936 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76331us
I20260709 12:17:48.372442 27001 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
W20260709 12:17:48.515329 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 120627us
I20260709 12:17:48.506076 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:48.526558 27088 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:17:48.571689 26344 raft_consensus.cc:1217] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 12:17:48.575686 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 129998us
W20260709 12:17:48.580544 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 186719us
W20260709 12:17:48.677480 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 292374us
W20260709 12:17:48.718360 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 90867.6us
W20260709 12:17:48.758687 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 113400us
I20260709 12:17:48.803438 27082 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:48.828048 25781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:48.828727 25781 raft_consensus.cc:606] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:48.832784 27068 tablet_copy_client.cc:806] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:48.833338 27068 tablet_copy_client.cc:670] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
W20260709 12:17:48.846374 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 94870.7us
W20260709 12:17:48.849924 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 101029us
I20260709 12:17:48.861038 27082 tablet_bootstrap.cc:492] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:48.904044 27082 ts_tablet_manager.cc:1403] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.686s	user 0.026s	sys 0.011s
I20260709 12:17:48.946147 27082 raft_consensus.cc:359] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:48.947150 27082 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:48.947750 27082 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:48.960978 27082 ts_tablet_manager.cc:1434] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.056s	user 0.003s	sys 0.004s
W20260709 12:17:48.964756 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 113866us
W20260709 12:17:48.976572 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 120232us
I20260709 12:17:48.989457 25796 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-b907e0d1516b4c78a2a977e29402bff9 received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:48.989975 25796 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-b907e0d1516b4c78a2a977e29402bff9 on tablet b907e0d1516b4c78a2a977e29402bff9 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:48.992631 27068 tablet_copy_client.cc:538] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:49.009060 27108 raft_consensus.cc:1064] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:49.012059 27108 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:49.013376 27113 raft_consensus.cc:993] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:49.014835 27101 raft_consensus.cc:1081] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:49.016554 25629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:49.017077 25629 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:49.017007 25782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:49.017740 25629 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:49.017938 25782 raft_consensus.cc:606] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:49.024735 25629 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:49.025748 27068 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:49.027386 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:49.027938 25707 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:49.036288 25707 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:49.037473 25603 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:49.038296 27020 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:49.038787 27020 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:49.039167 25632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:49.039880 25632 raft_consensus.cc:606] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:49.039669 27020 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:49.048156 25705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:49.048866 25705 raft_consensus.cc:606] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:49.050772 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:17:49.052454 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:49.057782 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:49.058642 25783 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:49.059058 25783 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:49.059916 25783 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:49.059474 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:49.069311 27102 raft_consensus.cc:1064] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:49.070673 25783 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:49.070720 25629 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:49.075693 25632 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:49.076432 25534 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:49.077106 25782 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:49.076431 26339 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:49.074112 27102 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:49.078344 27110 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:17:49.087348 27106 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:17:49.097954 27108 raft_consensus.cc:993] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:49.098649 27097 raft_consensus.cc:1081] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:49.098819 27116 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:49.102015 27068 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:49.102620 25782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:49.113093 27068 tablet_bootstrap.cc:492] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:49.113732 25782 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:49.114055 27068 ts_tablet_manager.cc:1403] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.088s	user 0.031s	sys 0.011s
I20260709 12:17:49.117671 27068 raft_consensus.cc:359] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:49.118865 27068 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:49.119812 27068 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:49.126083 27068 ts_tablet_manager.cc:1434] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:17:49.118112 25782 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.136806 25796 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-c3c8e03aa6454e118cd3f9f25c785bef received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:49.141396 25796 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-c3c8e03aa6454e118cd3f9f25c785bef on tablet c3c8e03aa6454e118cd3f9f25c785bef with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:49.147370 25629 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:49.152621 26336 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:49.155968 27116 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:49.165896 27102 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:49.167659 25785 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:49.169137 27102 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:49.188374 27125 raft_consensus.cc:993] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:49.188915 27125 raft_consensus.cc:1081] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:49.195067 25707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:17:49.195649 25707 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:49.195988 25707 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.198231 27126 raft_consensus.cc:993] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:49.216899 27126 raft_consensus.cc:1081] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:49.219457 25709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:17:49.220082 25709 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:49.220463 25709 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:49.221028 25707 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:49.225739 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:49.226433 25781 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:49.225798 25782 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:49.226778 25781 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:49.227598 25781 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:49.228727 25781 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.229727 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:49.230367 25705 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:49.230741 25705 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:49.224872 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:49.231426 26344 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.231585 25705 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:49.232620 25705 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.244715 26344 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:49.252863 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:49.260890 25635 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.245947 25783 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:49.262060 25707 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:49.263336 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:49.272523 25781 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:49.273784 25705 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:49.276678 25752 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:49.278417 25709 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:49.257351 27136 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:17:49.317354 25709 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:49.319444 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:49.320078 25631 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:49.320416 25631 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:49.321142 25631 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:49.324544 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53213.8us
I20260709 12:17:49.270634 25782 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:49.327924 25635 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:49.329190 25631 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:49.349862 26344 raft_consensus.cc:1217] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:49.309818 27049 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:49.350960 27049 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:49.351994 27049 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:49.314038 27108 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:49.367201 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:49.371731 25631 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:49.381002 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:17:49.373626 27132 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:17:49.385128 25782 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:49.408797 27008 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:49.420195 27008 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:49.421135 27008 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:49.384660 27108 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:17:49.437615 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 73025.8us
I20260709 12:17:49.487957 25785 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:49.507682 27133 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:17:49.501564 27001 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 12:17:49.546631 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 77900.7us
I20260709 12:17:49.494024 27108 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:49.426220 27102 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.564402 26336 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.569880 26339 raft_consensus.cc:1217] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:49.596486 25782 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:17:49.601297 25707 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:49.566079 27143 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.621646 25782 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.665009 25532 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:49.644507 27142 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
W20260709 12:17:49.719657 27144 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.113s	user 0.000s	sys 0.002s
I20260709 12:17:49.748636 26336 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.751518 27098 raft_consensus.cc:1064] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:49.775684 25631 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.770361 27098 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:49.780143 25783 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.825364 25533 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:49.845422 27147 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.166s	user 0.002s	sys 0.000s
I20260709 12:17:49.852394 25635 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:49.872997 25535 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:49.910715 27146 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:17:49.932358 25779 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:49.934513 25631 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:49.934309 27143 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:49.937057 27036 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:49.947139 25707 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:49.949126 27112 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:49.978092 25629 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:49.979852 27143 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:50.021433 26344 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:50.023547 27140 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:50.026608 25707 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:50.028276 27098 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:50.029570 27049 raft_consensus.cc:1064] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:50.032696 27049 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:50.039076 25629 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:50.041755 27098 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:17:50.135579 25534 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 248) took 1062 ms. Trace:
I20260709 12:17:50.135253 27115 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.139755 25785 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:50.143831 27097 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:50.146420 25707 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.136164 25534 rpcz_store.cc:276] 0709 12:17:49.073624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.073761 (+   137us) service_pool.cc:224] Handling call
0709 12:17:50.135485 (+1061724us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:49.388543 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:49.389306 (+   763us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:49.389359 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:17:49.389391 (+    32us) tablet.cc:662] Decoding operations
0709 12:17:49.389842 (+   451us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:49.399884 (+ 10042us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:49.410086 (+ 10202us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:49.435216 (+ 25130us) tablet.cc:701] Row locks acquired
0709 12:17:49.437728 (+  2512us) write_op.cc:260] PREPARE: finished
0709 12:17:49.463641 (+ 25913us) write_op.cc:270] Start()
0709 12:17:49.468093 (+  4452us) write_op.cc:276] Timestamp: P: 1783599469460838 usec, L: 0
0709 12:17:49.470916 (+  2823us) op_driver.cc:348] REPLICATION: starting
0709 12:17:49.526416 (+ 55500us) log.cc:844] Serialized 541 byte log entry
0709 12:17:49.753208 (+226792us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b18002c3920. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:17:49.886845 (+133637us) op_driver.cc:464] REPLICATION: finished
0709 12:17:50.112114 (+225269us) write_op.cc:301] APPLY: starting
0709 12:17:50.112188 (+    74us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:50.112266 (+    78us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:50.112280 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:50.112508 (+   228us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:50.112628 (+   120us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:50.112656 (+    28us) write_op.cc:312] APPLY: finished
0709 12:17:50.112819 (+   163us) log.cc:844] Serialized 23 byte log entry
0709 12:17:50.113012 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:50.113096 (+    84us) write_op.cc:454] Released schema lock
0709 12:17:50.113135 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:50.113495 (+   360us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:50.113531 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19470,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":225047,"apply.run_cpu_time_us":1369,"apply.run_wall_time_us":1369,"num_ops":1,"prepare.queue_time_us":305377,"prepare.run_cpu_time_us":19294,"prepare.run_wall_time_us":277205,"raft.queue_time_us":361255,"raft.run_cpu_time_us":57130,"raft.run_wall_time_us":220083,"replication_time_us":415765,"spinlock_wait_cycles":20875520,"thread_start_us":698701,"threads_started":4,"wal-append.queue_time_us":145940}]]}
I20260709 12:17:50.213052 26344 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:50.234633 27100 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:17:50.256456 25707 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:50.256647 25635 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:50.263145 26336 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:50.267318 25534 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.278308 25534 catalog_manager.cc:5697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:17:50.281867 25635 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.297080 26344 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.276836 27049 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:17:50.331830 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:50.334936 27036 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:50.337339 27140 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:50.377308 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 77427.3us
I20260709 12:17:50.381711 25534 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:50.606694 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 216419us
I20260709 12:17:50.621719 27052 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:50.632290 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 242839us
I20260709 12:17:50.666535 25705 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:50.672513 26344 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:50.674561 27110 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:50.694725 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 86816.3us
I20260709 12:17:50.724284 25784 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:50.734982 25779 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:50.741060 25705 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:50.767570 27097 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:17:50.823953 26336 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:50.839376 25707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:50.839995 25707 raft_consensus.cc:606] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:50.846901 26336 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.848480 27179 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:50.864807 25709 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:50.867467 27049 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:50.871488 26344 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:50.872171 27110 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.873106 25532 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:50.877128 25705 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.880538 25629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b74717dca3554e889066a717cfd1527a"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:50.881197 25629 raft_consensus.cc:606] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:50.890900 27104 raft_consensus.cc:993] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:50.891551 27159 raft_consensus.cc:1081] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
W20260709 12:17:50.891928 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 46939.5us
I20260709 12:17:50.894991 25635 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:50.897387 25782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:50.897935 25782 raft_consensus.cc:606] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:50.900952 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:50.888701 27186 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:17:50.903424 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:50.904560 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:50.908769 27190 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:17:50.911351 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 66501.8us
I20260709 12:17:50.916711 26344 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.932513 27186 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.938074 25705 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.947394 25629 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:50.961302 25779 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:50.980824 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b907e0d1516b4c78a2a977e29402bff9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:50.992204 25533 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.007544 27187 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.010808 25635 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.015918 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 31ccbf41d24e4275a7f921db24b940d4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:51.031644 26336 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:51.036616 25632 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:51.036813 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b74717dca3554e889066a717cfd1527a"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50982
I20260709 12:17:51.037422 25779 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:51.037818 25779 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:51.047143 25535 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 4 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.052297 25779 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.059689 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 31ccbf41d24e4275a7f921db24b940d4 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:51.064590 25532 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.081609 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 571c5e0767064f3e86479b794294d327 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:51.074126 25620 tablet_service.cc:1558] Processing DeleteTablet for tablet b907e0d1516b4c78a2a977e29402bff9 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:51.070572 27098 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:17:51.085861 27049 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:51.089385 26336 raft_consensus.cc:2955] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.100957 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:51.101699 25709 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:51.106277 27186 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.121795 26339 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.134531 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c3c8e03aa6454e118cd3f9f25c785bef with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:51.118367 25709 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:51.151937 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:51.153507 27049 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:51.154011 27049 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:51.165927 27049 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.179586 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 571c5e0767064f3e86479b794294d327 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:51.197381 25779 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:51.203382 25782 raft_consensus.cc:2955] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.225339 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:51.252833 25637 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:51.253198 25637 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:51.253961 25637 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.255228 25637 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:51.266826 25637 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:51.226646 27193 tablet_replica.cc:333] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:51.275609 27193 raft_consensus.cc:2243] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:51.276688 27193 raft_consensus.cc:2272] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:51.280471 27193 ts_tablet_manager.cc:1916] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:51.201545 27195 tablet_replica.cc:333] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:51.242663 25534 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:51.290515 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 86663.5us
I20260709 12:17:51.296267 27187 raft_consensus.cc:993] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:51.297199 27110 raft_consensus.cc:1081] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:51.316879 26344 raft_consensus.cc:2955] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.320508 25785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:51.321142 25785 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:51.323762 25785 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:51.335309 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.339867 27098 raft_consensus.cc:993] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:51.340389 27098 raft_consensus.cc:1081] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:51.353762 25638 raft_consensus.cc:2955] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:51.354372 25532 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:51.357475 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet c3c8e03aa6454e118cd3f9f25c785bef with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
W20260709 12:17:51.402045 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44568.4us
I20260709 12:17:51.419447 25632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:51.420195 25632 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:51.420610 25632 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:51.427131 27195 raft_consensus.cc:2243] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:51.429913 27193 ts_tablet_manager.cc:1929] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:17:51.430410 27193 log.cc:1199] T 31ccbf41d24e4275a7f921db24b940d4 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/31ccbf41d24e4275a7f921db24b940d4
I20260709 12:17:51.433111 27195 raft_consensus.cc:2272] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:51.435299 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 31ccbf41d24e4275a7f921db24b940d4 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:51.439046 27195 ts_tablet_manager.cc:1916] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:51.459440 27197 tablet_replica.cc:333] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:51.465842 27197 raft_consensus.cc:2243] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:51.468853 27197 raft_consensus.cc:2272] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:51.466609 27193 tablet_replica.cc:333] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:17:51.460280 25785 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.475860 27193 raft_consensus.cc:2243] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:51.480299 27193 raft_consensus.cc:2272] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:51.482793 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:51.483577 25638 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:51.473106 27195 ts_tablet_manager.cc:1929] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:51.466066 25632 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.496898 27195 log.cc:1199] T 571c5e0767064f3e86479b794294d327 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/571c5e0767064f3e86479b794294d327
I20260709 12:17:51.498196 25632 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:51.498624 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 571c5e0767064f3e86479b794294d327 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:51.500682 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:51.501318 25779 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:51.501642 25779 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:51.502517 25779 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.504750 25638 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:51.505311 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:51.505914 25705 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:51.506052 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:51.507081 27049 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:51.507529 27197 ts_tablet_manager.cc:1916] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:51.507601 27049 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:51.508773 27049 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.509478 25779 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:51.513326 25785 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:51.514256 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:51.514902 25709 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:51.515259 25709 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:51.516013 25709 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.517248 25709 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:51.524674 25709 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:51.527829 25705 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:51.533890 27197 ts_tablet_manager.cc:1929] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:51.538455 27197 log.cc:1199] T c3c8e03aa6454e118cd3f9f25c785bef P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/c3c8e03aa6454e118cd3f9f25c785bef
I20260709 12:17:51.534539 27193 ts_tablet_manager.cc:1916] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:51.540403 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet c3c8e03aa6454e118cd3f9f25c785bef (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:51.533341 25779 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:51.548302 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:51.549645 27104 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:51.550184 27104 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:51.551287 27104 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:51.563864 27193 ts_tablet_manager.cc:1929] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:51.564363 27193 log.cc:1199] T b907e0d1516b4c78a2a977e29402bff9 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/b907e0d1516b4c78a2a977e29402bff9
I20260709 12:17:51.567432 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet b907e0d1516b4c78a2a977e29402bff9 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:51.577944 25534 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:51.581571 25705 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:51.587174 27098 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:51.597244 25532 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:51.645377 25638 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:51.649523 27049 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:51.941468 25638 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:51.942767 27049 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:51.955765 25779 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:51.962816 27104 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:51.971324 25709 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:51.973109 27182 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.013764 25709 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:52.016791 27104 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:52.111712 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:52.153813 25632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:52.154568 25632 raft_consensus.cc:606] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:52.157421 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b74717dca3554e889066a717cfd1527a"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:52.157997 25785 raft_consensus.cc:606] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:52.160586 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:52.161264 25709 raft_consensus.cc:606] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:52.163821 25632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:52.164295 25632 raft_consensus.cc:606] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:17:52.165254 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:17:52.166356 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:52.166988 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:52.277987 27158 raft_consensus.cc:993] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:52.278618 27089 raft_consensus.cc:1081] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:17:52.280205 26336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:17:52.280642 26336 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.280874 26336 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.285097 26336 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:52.286845 26336 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:52.287550 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:52.288296 25638 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:52.288383 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:52.288686 25638 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:52.289034 25709 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.289529 25638 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:52.291760 25638 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.293744 25709 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:17:52.294718 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:52.295459 27040 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:52.298270 25638 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:17:52.301225 27040 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:17:52.302428 27040 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:52.312263 25532 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 3 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:52.399960 25632 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:52.407426 25785 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:52.407831 25601 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eb89b7cd7f4d4662ab9d1e1f55af650c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:52.408437 25709 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:52.408886 27104 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.409894 27089 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.418361 27089 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.420193 25709 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.421211 25785 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.441123 25532 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.441679 25632 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:52.460621 25709 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:52.461587 25785 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:52.461745 25601 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ff8dbae494348a7be4563e3bd2982b6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:52.461970 27159 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.463253 27158 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.471524 27159 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.476560 25709 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.480199 25779 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.488730 27190 raft_consensus.cc:993] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:52.489239 27190 raft_consensus.cc:1081] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:52.494920 25782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:52.495596 25782 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.496002 25782 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.500429 25532 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:52.501034 25785 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:52.503585 25782 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:52.521785 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:52.522476 25705 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:52.522846 25705 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:52.525240 25705 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:52.536443 25705 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.539448 25782 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:52.540472 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:52.541127 26336 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.548698 25705 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:52.550175 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:52.551088 27049 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:52.554418 25705 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:52.554927 25750 consensus_peers.cc:597] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 595303b3057546cbbfda4b84f8df8912. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:52.557514 27182 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.557958 25632 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:52.560751 26336 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:52.562108 27182 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.562897 27049 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:52.563768 27049 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:52.578393 25532 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:52.579083 27049 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.580823 25632 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.590826 25705 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.601330 25713 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:52.608505 25533 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:52.626698 25632 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:52.627707 25782 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:52.629021 25676 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: afe9ac51e0a949a7a43e288f3d2ed652. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:52.629586 27179 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:52.631695 27190 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:52.663297 27190 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.665629 25782 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.666030 25632 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.684255 25534 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:52.723242 27159 raft_consensus.cc:993] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:52.723806 27104 raft_consensus.cc:1081] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:52.725613 25713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:17:52.726117 25713 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.726254 27226 raft_consensus.cc:993] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:52.726472 25713 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.726877 27182 raft_consensus.cc:1081] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:52.728533 25632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b74717dca3554e889066a717cfd1527a"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:52.729051 25632 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.729395 25632 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.734094 25713 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:52.735472 25632 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:52.736995 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:52.737522 25638 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:52.737800 25638 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:52.738461 25638 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:52.739463 25638 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.739972 25713 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:52.742419 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:52.743494 25782 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.747460 25638 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:52.748693 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:52.749548 27190 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:52.750114 27190 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:52.751006 27190 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:52.751278 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:52.752351 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:52.752555 25713 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.752903 25785 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:52.753212 25785 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:52.751495 25632 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:52.760319 25782 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:52.754216 25785 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:52.761133 25713 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:17:52.762236 25603 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:52.763100 27104 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:52.763588 27104 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:52.764356 27104 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:52.770458 25532 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:52.762236 25785 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.783564 25785 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:17:52.790899 25632 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:52.793179 27210 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:52.795899 25532 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 3 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:52.843839 25709 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:52.846134 27210 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:52.900115 27240 ts_tablet_manager.cc:933] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:52.900606 27241 ts_tablet_manager.cc:933] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:17:52.911656 27241 tablet_copy_client.cc:323] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:52.916071 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet eb89b7cd7f4d4662ab9d1e1f55af650c from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:52.916730 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet eb89b7cd7f4d4662ab9d1e1f55af650c from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-eb89b7cd7f4d4662ab9d1e1f55af650c
I20260709 12:17:52.918756 27240 tablet_copy_client.cc:323] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:17:52.927168 25647 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet 4ff8dbae494348a7be4563e3bd2982b6 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:17:52.927739 25647 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet 4ff8dbae494348a7be4563e3bd2982b6 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-4ff8dbae494348a7be4563e3bd2982b6
I20260709 12:17:52.948460 25647 tablet_copy_source_session.cc:215] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:52.948585 25648 tablet_copy_source_session.cc:215] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:52.952472 27241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb89b7cd7f4d4662ab9d1e1f55af650c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:52.953296 27240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff8dbae494348a7be4563e3bd2982b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:52.971107 27241 tablet_copy_client.cc:806] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:52.971765 27241 tablet_copy_client.cc:670] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:52.972710 27240 tablet_copy_client.cc:806] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:52.973273 27240 tablet_copy_client.cc:670] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:52.977193 27241 tablet_copy_client.cc:538] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:52.983134 27240 tablet_copy_client.cc:538] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:52.990157 27241 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:53.000622 25713 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:53.002851 27240 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:53.005671 27222 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:53.021988 27241 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:53.023252 27241 tablet_bootstrap.cc:492] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:53.023993 27241 ts_tablet_manager.cc:1403] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.034s	user 0.028s	sys 0.005s
I20260709 12:17:53.027871 27241 raft_consensus.cc:359] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.028796 27241 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:53.029553 27241 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.031826 27241 ts_tablet_manager.cc:1434] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:17:53.041430 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-eb89b7cd7f4d4662ab9d1e1f55af650c received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:53.041900 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-eb89b7cd7f4d4662ab9d1e1f55af650c on tablet eb89b7cd7f4d4662ab9d1e1f55af650c with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:53.050734 27240 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:53.051789 27240 tablet_bootstrap.cc:492] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:53.052511 27240 ts_tablet_manager.cc:1403] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.050s	user 0.021s	sys 0.012s
I20260709 12:17:53.055904 27240 raft_consensus.cc:359] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.056761 27240 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:53.057422 27240 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.060032 27240 ts_tablet_manager.cc:1434] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:53.062687 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-4ff8dbae494348a7be4563e3bd2982b6 received from {username='slave'} at 127.0.0.1:60736
I20260709 12:17:53.063172 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-4ff8dbae494348a7be4563e3bd2982b6 on tablet 4ff8dbae494348a7be4563e3bd2982b6 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:53.078068 26344 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:53.079928 27182 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:53.161254 27240 ts_tablet_manager.cc:933] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:53.168354 27240 tablet_copy_client.cc:323] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:53.170971 25713 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:53.171159 25796 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet 595303b3057546cbbfda4b84f8df8912 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:53.171857 25796 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet 595303b3057546cbbfda4b84f8df8912 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-595303b3057546cbbfda4b84f8df8912
I20260709 12:17:53.172552 27104 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:53.180361 25796 tablet_copy_source_session.cc:215] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:53.183966 25785 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:53.185459 27159 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:53.186096 27240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595303b3057546cbbfda4b84f8df8912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:53.203791 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:53.229717 27240 tablet_copy_client.cc:806] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:53.230325 27240 tablet_copy_client.cc:670] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:53.236220 27240 tablet_copy_client.cc:538] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:53.245131 27240 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:53.249034 27241 ts_tablet_manager.cc:933] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:53.265401 25782 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:53.267606 27241 tablet_copy_client.cc:323] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:17:53.269649 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet afe9ac51e0a949a7a43e288f3d2ed652 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:17:53.267520 27228 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:53.273792 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet afe9ac51e0a949a7a43e288f3d2ed652 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-afe9ac51e0a949a7a43e288f3d2ed652
I20260709 12:17:53.284466 25723 tablet_copy_source_session.cc:215] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:53.297705 27241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe9ac51e0a949a7a43e288f3d2ed652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:53.331087 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:53.331785 25709 raft_consensus.cc:606] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:53.335182 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:53.335875 25785 raft_consensus.cc:606] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:53.339891 25635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:53.340541 25635 raft_consensus.cc:606] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:53.364631 25638 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:53.364898 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:53.365509 25785 raft_consensus.cc:606] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:53.367372 27228 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:53.369042 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:17:53.370982 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:53.371738 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:53.380662 27240 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:53.384066 27240 tablet_bootstrap.cc:492] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:53.384244 26344 raft_consensus.cc:1217] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:53.384824 27240 ts_tablet_manager.cc:1403] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.140s	user 0.066s	sys 0.047s
I20260709 12:17:53.387744 27240 raft_consensus.cc:359] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.388574 27240 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:53.389171 27240 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.383067 27241 tablet_copy_client.cc:806] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:53.393168 27241 tablet_copy_client.cc:670] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:53.395694 27240 ts_tablet_manager.cc:1434] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:17:53.396265 26344 raft_consensus.cc:1217] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:53.412551 25796 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-595303b3057546cbbfda4b84f8df8912 received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:53.413050 25796 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-595303b3057546cbbfda4b84f8df8912 on tablet 595303b3057546cbbfda4b84f8df8912 with peer 10d0c7349085463595aedaf7c071e742
W20260709 12:17:53.419153 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44460.5us
I20260709 12:17:53.447803 27241 tablet_copy_client.cc:538] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:53.458474 27241 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:53.494141 27241 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:53.496786 27241 tablet_bootstrap.cc:492] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:53.497437 27241 ts_tablet_manager.cc:1403] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.004s
I20260709 12:17:53.500330 27241 raft_consensus.cc:359] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.501169 27241 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:53.501832 27241 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:53.504321 27241 ts_tablet_manager.cc:1434] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:17:53.506261 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-afe9ac51e0a949a7a43e288f3d2ed652 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:17:53.506686 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-afe9ac51e0a949a7a43e288f3d2ed652 on tablet afe9ac51e0a949a7a43e288f3d2ed652 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:53.515501 27222 raft_consensus.cc:993] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:53.516034 27222 raft_consensus.cc:1081] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:53.517827 25635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:53.518389 25635 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:53.518762 25635 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.525482 25635 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:53.528002 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:53.528223 25635 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:53.528564 25709 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.528914 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:53.529557 25785 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:53.529845 25785 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:53.531204 25785 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:53.532178 25785 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.534912 25709 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:17:53.536178 25603 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:53.537315 27251 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:53.551159 25785 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:17:53.552402 27251 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:53.553455 27251 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:53.565476 25532 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 3 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:53.641314 27222 raft_consensus.cc:993] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:53.641877 27098 raft_consensus.cc:1081] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:53.643551 25709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:17:53.644151 25709 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:53.644541 25709 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.652864 25709 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.657464 25709 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:53.662884 27257 raft_consensus.cc:993] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:53.662649 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:53.663446 27257 raft_consensus.cc:1081] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:53.663568 25785 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:53.664113 25785 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:53.666097 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:53.666361 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:53.665081 25785 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.666764 25779 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:53.666962 26344 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.667178 25779 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.667517 25785 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.674775 26344 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:53.676537 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:53.677525 25779 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.682003 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:53.682561 25709 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:53.678439 27258 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:53.682866 25709 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:53.683141 27258 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:53.683450 25779 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:53.683607 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:53.684227 26336 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.684023 27258 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.678463 25785 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:53.687235 25709 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.692708 25709 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.753798 25709 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
W20260709 12:17:53.770929 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 68746.5us
W20260709 12:17:53.778909 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 76339.5us
I20260709 12:17:53.779455 26344 raft_consensus.cc:1217] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:53.785429 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:17:53.846990 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61992.1us
I20260709 12:17:53.862411 27098 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:53.863353 27098 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:53.865157 27098 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.912067 26342 raft_consensus.cc:1217] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:53.944276 27159 raft_consensus.cc:1064] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:53.948830 27159 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.956187 27104 raft_consensus.cc:1064] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:53.955996 25532 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:53.954035 26336 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:53.959729 27104 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:53.974460 27266 raft_consensus.cc:993] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:53.975664 27266 raft_consensus.cc:1081] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
W20260709 12:17:54.025869 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48750.5us
I20260709 12:17:54.052562 25713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:17:54.054570 25713 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:54.054984 25713 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:54.067520 25707 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:54.075881 27274 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:54.095170 25707 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.106702 25779 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:54.109378 26342 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.115060 25779 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.116148 27266 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.116930 27251 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:54.118247 27251 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.129305 26336 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.131300 25713 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:54.135531 25713 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:54.148262 25784 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.164690 27104 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:54.169912 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:54.170967 25783 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:54.177096 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:54.177827 25635 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:54.178319 25635 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:54.180621 25707 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.196201 25783 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:17:54.179975 27159 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:17:54.200778 25635 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:54.234915 25635 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:54.236009 26336 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:54.242002 27190 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:54.249876 25635 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:17:54.267086 27104 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:17:54.272651 27266 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:17:54.279484 27272 raft_consensus.cc:1064] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:54.321439 25677 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:54.322919 27257 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:54.323624 27257 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:54.324615 27257 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:17:54.334983 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 108333us
I20260709 12:17:54.346794 27268 raft_consensus.cc:1064] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:17:54.349843 27268 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:54.338936 25532 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:54.359768 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 82062.3us
W20260709 12:17:54.364656 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 137803us
W20260709 12:17:54.452184 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 67535.9us
I20260709 12:17:54.468318 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:54.503710 27297 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:17:54.523311 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.504405 27298 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:54.575232 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.581712 27280 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:17:54.589144 26344 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:54.591312 27290 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:54.611958 25709 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:54.633044 26336 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:17:54.704771 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50326.8us
W20260709 12:17:54.757637 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 91293.5us
I20260709 12:17:54.753460 26336 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:54.770882 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61711.6us
W20260709 12:17:54.774439 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 118015us
W20260709 12:17:54.783947 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 125334us
W20260709 12:17:54.787400 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.790656 27314 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 12:17:54.804418 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45713.4us
I20260709 12:17:54.806989 27291 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:54.808192 25713 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:54.816617 27272 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:54.824926 27310 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:17:54.787847 27302 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.177s
W20260709 12:17:54.852162 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 84568.7us
W20260709 12:17:54.853394 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76511.6us
I20260709 12:17:54.872296 25784 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:54.879998 25636 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:17:54.892361 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.015s	sys 0.003s
W20260709 12:17:54.892797 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 505270us
I20260709 12:17:54.913548 25709 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:54.940668 25637 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.956727 25533 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:54.960639 25785 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:54.965783 25781 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.968312 27291 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:17:54.968796 27323 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:54.969205 26336 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.971843 25707 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:54.975576 25533 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:54.979653 27098 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.979077 25781 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.002650 27291 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:55.005807 27291 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:55.008746 27272 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:55.011165 27319 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:55.011511 25638 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:55.013201 27257 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:55.026052 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:17:55.031461 25705 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:55.035166 25786 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:55.036394 27104 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:55.010074 25532 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 4, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 4 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:55.064548 25636 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:55.102717 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:17:55.105389 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.114838 26336 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:55.108675 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:55.032897 27257 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:17:55.129487 27279 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:55.156167 27274 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:55.158742 27298 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.182528 26339 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:55.185544 27333 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 12:17:55.205493 27327 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:17:55.198733 25705 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:17:55.237265 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 56542.8us
W20260709 12:17:55.269925 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 67518.7us
I20260709 12:17:55.244683 26344 raft_consensus.cc:2955] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.310931 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ff8dbae494348a7be4563e3bd2982b6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:55.317831 27312 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.321146 27098 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.335747 25705 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.372025 27286 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:17:55.396953 26339 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.472518 25785 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.473167 25632 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.495582 25636 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.496853 25533 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.496976 26339 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.503261 25532 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.544719 25637 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:55.554214 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ff8dbae494348a7be4563e3bd2982b6 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:55.579169 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:55.579903 25709 raft_consensus.cc:606] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:55.584411 27347 tablet_replica.cc:333] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:55.585676 27347 raft_consensus.cc:2243] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:55.586889 27347 raft_consensus.cc:2272] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:55.588634 25786 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:55.589201 25786 raft_consensus.cc:606] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:55.578577 25532 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:55.591970 27347 ts_tablet_manager.cc:1916] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:55.592530 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:17:55.593133 25636 raft_consensus.cc:606] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:55.594906 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:55.596645 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:55.597716 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:55.609609 27347 ts_tablet_manager.cc:1929] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:55.610260 27347 log.cc:1199] T 4ff8dbae494348a7be4563e3bd2982b6 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/4ff8dbae494348a7be4563e3bd2982b6
I20260709 12:17:55.610896 26344 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:55.610934 25786 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:55.613610 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 4ff8dbae494348a7be4563e3bd2982b6 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:55.632792 27159 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:55.634702 27292 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:17:55.647850 25520 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leader transfer in progress [suppressed 4 similar messages]
I20260709 12:17:55.650584 25709 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:55.663184 27292 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.672436 25637 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:55.674593 27267 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:55.677852 25785 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.682199 26344 raft_consensus.cc:2955] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.687083 27268 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.693542 25636 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.688102 26339 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:55.697854 27312 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:17:55.695513 25520 catalog_manager.cc:4755] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): ChangeConfig:REMOVE_PEER RPC failed for tablet 595303b3057546cbbfda4b84f8df8912: Remote error: Service unavailable: leader transfer in progress
I20260709 12:17:55.705420 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eb89b7cd7f4d4662ab9d1e1f55af650c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:55.705086 26342 raft_consensus.cc:2955] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.728315 25532 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 4, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 1 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.732414 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet afe9ac51e0a949a7a43e288f3d2ed652 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:55.751959 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet eb89b7cd7f4d4662ab9d1e1f55af650c with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57962
I20260709 12:17:55.751801 25535 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 3 to 4, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 1 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:17:55.783694 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet afe9ac51e0a949a7a43e288f3d2ed652 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50958
I20260709 12:17:55.784932 27347 tablet_replica.cc:333] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:17:55.786069 27347 raft_consensus.cc:2243] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:55.786974 27347 raft_consensus.cc:2272] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:55.796805 27349 tablet_replica.cc:333] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:17:55.798560 27349 raft_consensus.cc:2243] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:55.800625 27349 raft_consensus.cc:2272] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:55.803633 27347 ts_tablet_manager.cc:1916] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:55.815826 27349 ts_tablet_manager.cc:1916] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:55.877095 27347 ts_tablet_manager.cc:1929] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:55.877679 27347 log.cc:1199] T afe9ac51e0a949a7a43e288f3d2ed652 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/afe9ac51e0a949a7a43e288f3d2ed652
I20260709 12:17:55.879525 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet afe9ac51e0a949a7a43e288f3d2ed652 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:55.900956 27349 ts_tablet_manager.cc:1929] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:55.901618 27349 log.cc:1199] T eb89b7cd7f4d4662ab9d1e1f55af650c P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/eb89b7cd7f4d4662ab9d1e1f55af650c
I20260709 12:17:55.903731 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet eb89b7cd7f4d4662ab9d1e1f55af650c (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:17:55.923276 27301 raft_consensus.cc:993] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:55.923954 27303 raft_consensus.cc:1081] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:55.928516 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:55.929400 25636 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:55.929762 25636 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.937736 25636 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:55.942440 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:55.943118 26342 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.944797 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:55.945479 25705 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.953761 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:55.955192 25786 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:55.955526 25786 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:55.954046 26342 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:55.954612 25636 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:55.956432 25786 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:55.960554 25786 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.969321 25786 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:55.970687 25601 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:55.971645 27291 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:55.972150 27291 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:55.973124 27291 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:55.975868 25705 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:56.020233 25534 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:56.036633 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:56.037129 25636 raft_consensus.cc:606] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Received request to transfer leadership
I20260709 12:17:56.040146 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:56.040650 25636 raft_consensus.cc:606] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Received request to transfer leadership
I20260709 12:17:56.043200 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:17:56.043630 25636 raft_consensus.cc:606] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Received request to transfer leadership
W20260709 12:17:56.047986 25584 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:17:56.048333 25584 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 595303b3057546cbbfda4b84f8df8912 already in progress
W20260709 12:17:56.048569 25584 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 595303b3057546cbbfda4b84f8df8912 already in progress
I20260709 12:17:56.073699 27284 raft_consensus.cc:993] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.074242 27323 raft_consensus.cc:1081] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.075731 25786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:56.076310 25786 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:56.076617 25786 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:56.083858 25786 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.085434 25786 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:56.086120 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:56.086530 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:56.087023 25637 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:56.087144 25709 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:56.087604 25709 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:56.088274 25709 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.089298 25709 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:56.094244 25637 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:56.094915 25709 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:56.095770 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:56.096570 27303 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:56.096979 27303 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:56.097801 27303 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.108888 25534 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:56.147431 27292 raft_consensus.cc:993] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.148114 27291 raft_consensus.cc:1081] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.150034 25786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50982
I20260709 12:17:56.150662 25786 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:56.150945 25786 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:56.159439 25786 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.161399 25786 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:56.162140 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:56.162392 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:56.163050 25637 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:56.163182 25709 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:56.163345 25637 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:56.164098 25637 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.165030 25637 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:56.169096 25709 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:56.171517 25637 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:17:56.172232 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:56.173805 27303 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:56.177743 27303 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:56.179050 27303 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.189145 25534 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:56.489404 25786 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:56.491299 27292 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:56.496232 27291 raft_consensus.cc:993] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.496717 27292 raft_consensus.cc:1081] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.498921 25786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50982
I20260709 12:17:56.499517 25786 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:56.499909 25786 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.507553 25786 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:56.509529 25786 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:56.510171 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:56.510540 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:17:56.511170 25705 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.511296 26342 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.510226 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:56.512559 25636 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:56.512861 25636 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:56.513669 25636 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:56.515025 25636 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.517869 25705 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:56.518579 25709 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:56.522605 25636 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:56.523193 26342 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:17:56.523763 25751 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:56.524633 27303 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:56.525141 27303 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:56.525992 27303 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:56.537024 25534 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:56.564352 25705 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:56.565508 27098 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:56.575685 25636 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:56.576928 27098 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:56.666882 25705 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:56.668501 27098 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:56.680220 25637 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:56.683260 27098 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:56.808691 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:56.854931 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:56.855505 25785 raft_consensus.cc:606] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:56.858367 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:56.859046 25785 raft_consensus.cc:606] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:17:56.861495 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:17:56.862021 25785 raft_consensus.cc:606] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:17:56.864539 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
I20260709 12:17:56.865206 25709 raft_consensus.cc:606] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:17:56.866554 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:17:56.867904 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:56.868551 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:56.868737 25520 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 595303b3057546cbbfda4b84f8df8912 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 955233c139344131a847044f17f8d348 (127.23.217.67:41061) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:56.944891 27284 raft_consensus.cc:993] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.945631 27323 raft_consensus.cc:1081] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:17:56.947769 25785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:17:56.948882 25785 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:56.949276 25785 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.956297 25785 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.959757 25785 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:56.959789 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:56.959805 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:56.960775 25709 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:56.960841 25636 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.961283 25709 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:56.962917 25709 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.964160 25709 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.967839 25636 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:17:56.968950 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:56.969767 27098 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.970280 27098 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:56.970474 25709 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:17:56.971163 27098 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:56.979972 25636 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:56.981631 27098 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:56.982059 25533 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 4 to 5, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 5 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:57.005409 25705 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:57.006601 27303 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:57.017148 26342 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:57.018965 27301 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:57.132650 25785 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.138264 25705 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:17:57.138303 25750 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60051e5723b2429582407076a003096c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.138218 25636 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:57.140182 27303 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.140993 27364 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.146996 27098 raft_consensus.cc:2955] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.148849 25636 raft_consensus.cc:2955] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.148993 25705 raft_consensus.cc:2955] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.161664 25785 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.168404 25535 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 3, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.181666 25636 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:57.181689 25705 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:57.183362 27303 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:57.184674 25750 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a1f6623a51a47759e59269cfa63be4b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.185384 27364 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:57.193579 27364 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.206454 27098 raft_consensus.cc:993] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:57.206981 27098 raft_consensus.cc:1081] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:17:57.210557 25637 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.214748 25709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:17:57.215351 25709 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:57.215718 25709 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:57.219933 25533 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:57.229092 25709 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.235989 25785 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.236446 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:57.237140 25786 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:57.237471 25786 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:57.238273 25786 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.239374 25786 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:57.239394 25705 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.243264 25709 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:57.250138 25786 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:57.251492 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:57.252499 27323 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:57.253046 27323 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:17:57.255570 27323 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.260849 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:17:57.261479 25632 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:57.276360 25707 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:57.276535 25638 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:57.278227 25750 consensus_peers.cc:597] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7701d2ef98204ad0bedc9231fe829f69. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.280640 27098 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:57.282397 25533 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:57.286417 27364 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:57.289435 27098 raft_consensus.cc:993] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:57.290099 27364 raft_consensus.cc:1081] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:57.292744 25638 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:57.293533 25638 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:57.294011 25638 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:57.297036 27098 raft_consensus.cc:993] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:57.298237 27098 raft_consensus.cc:1081] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:17:57.304572 25632 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:17:57.305965 25638 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.307953 25635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:17:57.310639 25635 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:57.311026 25635 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:57.326435 25635 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.330018 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:57.330621 25709 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:57.331040 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:57.331732 25786 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:57.331277 27303 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.332043 25786 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:57.333962 25786 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.335391 25786 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:57.341147 25709 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:57.346809 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:17:57.348517 27292 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:57.349211 27292 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:57.350621 27292 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.357035 25712 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:57.359118 27364 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:57.360092 27364 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:57.360419 27364 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:17:57.361243 27364 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.327013 25638 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:57.367851 25635 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:57.368983 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:17:57.362365 27364 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:57.375249 25786 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:57.371641 25635 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.378110 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:17:57.381196 27355 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:57.324590 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:17:57.382588 25705 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:57.429617 25707 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.434331 25533 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:57.442842 27355 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:17:57.444043 27355 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:57.454414 25705 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:17:57.467116 25637 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.475569 25638 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:57.465894 25532 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 2, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:57.477895 27375 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:17:57.484853 25782 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
W20260709 12:17:57.508787 25601 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef4fd24cfa184aa8a8a1de8f32296e6a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.511317 25782 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:57.511013 25533 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:57.506965 25707 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:57.514063 27379 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:57.520984 25705 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:57.521255 27375 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:57.522609 27355 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:57.560637 27379 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.564343 25782 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.564343 25707 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.589782 25533 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 5, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:17:57.612689 27389 ts_tablet_manager.cc:933] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:57.632251 27389 tablet_copy_client.cc:323] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:57.635277 25796 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet 60051e5723b2429582407076a003096c from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:57.635999 25796 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet 60051e5723b2429582407076a003096c from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-60051e5723b2429582407076a003096c
I20260709 12:17:57.649911 25796 tablet_copy_source_session.cc:215] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:57.653327 27389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60051e5723b2429582407076a003096c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:57.670046 27389 tablet_copy_client.cc:806] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:57.670696 27389 tablet_copy_client.cc:670] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:57.675246 27389 tablet_copy_client.cc:538] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:57.685834 27389 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:57.698140 27391 ts_tablet_manager.cc:933] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:57.710645 27391 tablet_copy_client.cc:323] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:17:57.712872 25796 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet 8a1f6623a51a47759e59269cfa63be4b from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:17:57.713442 25796 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet 8a1f6623a51a47759e59269cfa63be4b from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-8a1f6623a51a47759e59269cfa63be4b
I20260709 12:17:57.719957 27389 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:57.721251 25796 tablet_copy_source_session.cc:215] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:57.721376 27389 tablet_bootstrap.cc:492] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:57.722574 27389 ts_tablet_manager.cc:1403] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.037s	user 0.032s	sys 0.006s
I20260709 12:17:57.725030 27391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a1f6623a51a47759e59269cfa63be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:57.726678 27389 raft_consensus.cc:359] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.727607 27389 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:57.728271 27389 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.731180 27389 ts_tablet_manager.cc:1434] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:17:57.735564 25796 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-60051e5723b2429582407076a003096c received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:57.736147 25796 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-60051e5723b2429582407076a003096c on tablet 60051e5723b2429582407076a003096c with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:57.741856 25635 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:57.747354 27391 tablet_copy_client.cc:806] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:57.753288 27391 tablet_copy_client.cc:670] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:57.753141 27373 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.766561 27391 tablet_copy_client.cc:538] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:57.778877 27391 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:17:57.779444 25785 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:57.782670 27374 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.842741 27391 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:57.843891 27391 tablet_bootstrap.cc:492] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:17:57.844640 27391 ts_tablet_manager.cc:1403] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.000s
I20260709 12:17:57.848241 27391 raft_consensus.cc:359] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.849315 27391 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:17:57.850060 27391 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:17:57.857589 27391 ts_tablet_manager.cc:1434] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:17:57.861734 25796 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-8a1f6623a51a47759e59269cfa63be4b received from {username='slave'} at 127.0.0.1:50194
I20260709 12:17:57.862200 25796 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-8a1f6623a51a47759e59269cfa63be4b on tablet 8a1f6623a51a47759e59269cfa63be4b with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:17:57.865571 25709 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:57.879303 27292 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:57.901397 27391 ts_tablet_manager.cc:933] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:17:57.908691 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:59.023891 27401 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:59.024710 27401 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:59.026661 27400 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:59.190438 27399 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:17:59.191547 27399 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:59.194518 27399 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:59.195264 27399 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:59.203475 27399 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:59.301069 27401 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:59.041795 27400 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:17:59.450196 27399 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:59.452488 27400 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:17:59.453622 27402 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:59.454378 27401 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:59.453768 27400 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:59.455184 27399 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:17:59.454658 27402 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:59.455529 27400 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:59.455871 27399 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:59.455057 27401 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:59.457808 27400 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:59.458518 27399 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:59.458873 27401 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:17:59.459338 27402 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:17:59.575790 25567 debug-util.cc:398] Leaking SignalData structure 0x7b08000c7d60 after lost signal to thread 24424
I20260709 12:17:59.698591 27402 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:17:59.699535 27402 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:17:59.702232 27402 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:00.107429 27391 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 2.205s	user 0.017s	sys 0.038s
W20260709 12:18:00.107931 27391 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 2.206s	user 0.017s	sys 0.038s
I20260709 12:18:00.108807 27391 tablet_copy_client.cc:323] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.67:41061
I20260709 12:18:00.110033 27404 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.111451 27403 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.111995 27403 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.114616 27403 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.115701 27406 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.119944 27406 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.111124 27404 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.124385 27404 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.125312 27406 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.127449 27408 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.128106 27408 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.130795 27408 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.131274 27408 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.131793 27408 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.133708 27408 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.134156 27408 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.134573 27408 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.136451 27408 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.136914 27408 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.137346 27408 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.138634 27405 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.139195 27405 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.150789 25782 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:00.151777 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1798) took 1172 ms. Trace:
I20260709 12:18:00.152393 27405 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.152256 25782 rpcz_store.cc:276] 0709 12:17:58.979331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.149747 (+1170416us) service_pool.cc:224] Handling call
0709 12:18:00.150259 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.151401 (+  1142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.151698 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:00.155656 27408 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.183542 25533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 298) took 2347 ms. Trace:
I20260709 12:18:00.186883 25533 rpcz_store.cc:276] 0709 12:17:57.836392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.893501 (+ 57109us) service_pool.cc:224] Handling call
0709 12:18:00.183397 (+2289896us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:18:00.190372 27407 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.194811 27407 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.197522 27407 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.191578 25535 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 293) took 2257 ms. Trace:
I20260709 12:18:00.198388 25535 rpcz_store.cc:276] 0709 12:17:57.933962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.934144 (+   182us) service_pool.cc:224] Handling call
0709 12:18:00.191509 (+2257365us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:18:00.213050 27412 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.217486 27412 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.227427 27411 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.228289 27411 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.231572 27411 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.234543 27412 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.243294 27416 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.259338 27406 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.259977 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1797) took 1281 ms. Trace:
I20260709 12:18:00.259883 27406 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.260318 25786 rpcz_store.cc:276] 0709 12:17:58.978949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.259206 (+1280257us) service_pool.cc:224] Handling call
0709 12:18:00.259512 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.259677 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.259704 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:00.259745 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:00.259771 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.259818 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.259916 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:00.261888 27406 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.262369 27406 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.266019 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:00.251667 27411 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.299487 27413 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.299397 27418 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.300216 27413 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.300312 27418 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.277880 25796 tablet_copy_service.cc:140] P 955233c139344131a847044f17f8d348: Received BeginTabletCopySession request for tablet 7701d2ef98204ad0bedc9231fe829f69 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50194)
I20260709 12:18:00.262887 27406 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.264324 27416 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.333213 25796 tablet_copy_service.cc:161] P 955233c139344131a847044f17f8d348: Beginning new tablet copy session on tablet 7701d2ef98204ad0bedc9231fe829f69 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50194: session id = 10d0c7349085463595aedaf7c071e742-7701d2ef98204ad0bedc9231fe829f69
I20260709 12:18:00.331561 27413 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.319908 27411 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.339219 27419 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.374085 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.374790 25711 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:00.374981 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.375543 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.375788 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.376438 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.376573 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.377175 25708 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:00.377172 25706 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:00.378232 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.378602 27420 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.378981 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.379581 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.379331 27420 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:00.380437 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.380707 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.381269 25710 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:00.381381 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.382678 27405 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:00.383095 27405 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.383453 27405 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.383888 27416 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.378216 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.385758 25708 raft_consensus.cc:2410] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:18:00.385603 27406 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.376451 25711 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:00.388996 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.378330 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.390174 25706 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:00.355403 27407 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.356681 27413 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.396369 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:00.397292 25786 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:00.398967 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:00.401844 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.404143 27428 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.405141 27428 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.406543 27409 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:00.407157 27409 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.407681 27409 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.408198 27428 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.408949 27428 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:00.409387 27428 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.410015 27428 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:00.369653 27423 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.388278 27403 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:00.370141 27415 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.386205 27414 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.392202 27402 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:00.393954 27408 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:00.394055 27410 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.440831 27420 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.445387 27405 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.445748 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:00.415737 27413 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.437908 27414 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.448378 27413 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.448578 27414 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.448920 27413 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.431643 27423 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.449391 27413 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.450968 27423 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.451303 27413 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.451483 27423 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.451819 27413 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.451961 27423 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.452310 27413 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.453083 27418 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.453974 27423 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.454198 27413 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.454511 27411 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.438079 27415 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.389014 27419 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.433185 27403 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.436599 27402 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.413870 27407 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.440145 27408 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.463456 27418 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.470018 27402 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.468420 27403 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.469084 27423 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.486037 27408 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.470207 27411 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.488546 27416 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.456012 27410 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.485271 27418 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.534685 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:00.537137 25638 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:00.532433 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.538753 25713 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:00.528723 25785 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.537703 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.529727 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:00.529582 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:00.506031 27412 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.542433 25633 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:00.503726 27421 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.559877 27440 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.560889 27440 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.503700 27431 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.519742 27423 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.565801 25629 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:00.565996 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:00.504616 27435 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.570124 25711 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.531086 27407 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.575397 27433 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.529072 27415 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.533720 27416 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.576030 27433 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.533502 27419 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:00.572513 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 107442us
I20260709 12:18:00.525022 27405 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.565898 27437 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.563753 27427 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.547216 27411 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.585594 27416 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.586139 27416 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.586315 27429 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.586570 27411 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.586935 27429 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.587291 27411 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:00.588608 27440 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.589217 27423 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.589277 27429 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.586665 27416 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.542935 27409 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.592784 25711 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
W20260709 12:18:00.593154 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 54853.7us
I20260709 12:18:00.587210 27432 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.571382 27431 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.567812 27412 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.590760 27422 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.567371 27421 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.599818 27416 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.600319 27407 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.628283 27438 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:00.626760 27422 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.631942 27438 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.632555 27438 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:00.628629 27441 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.594324 27427 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.599915 27410 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.591763 27437 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.633816 27422 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.577452 27435 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:00.633371 27441 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.626181 25796 tablet_copy_source_session.cc:215] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:18:00.623502 27420 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.634599 27416 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:00.653577 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 119657us
I20260709 12:18:00.642584 27413 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.703370 27410 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.639707 27423 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:00.642084 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 103616us
I20260709 12:18:00.640173 27406 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.639091 27407 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.715881 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:00.726692 27447 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.727443 27447 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.735502 27448 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.736231 27448 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.751019 27449 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.751956 27449 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:00.692628 27418 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.755740 27440 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.756474 27440 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.764138 25709 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:00.769840 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:00.706353 27416 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.771723 27431 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.772336 27431 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.620839 27432 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.772876 27431 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.716238 27410 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.707293 27420 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.701181 27413 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:00.754946 27424 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:00.766345 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:00.676731 27422 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.785004 26339 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:00.770659 26342 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:00.811163 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:00.811712 27448 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.812817 27426 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.813644 27426 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.815032 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:00.821972 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.809971 27427 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.746783 27433 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:00.832158 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 131769us
I20260709 12:18:00.833492 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:00.835244 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.835901 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:00.836629 25629 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
W20260709 12:18:00.837602 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 128395us
I20260709 12:18:00.840024 25752 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.771783 27439 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:00.713155 27423 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.720283 27406 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:00.845590 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:00.853099 27451 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:00.854385 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.842022 27427 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.845832 27416 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.840694 27415 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:00.825099 27422 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.841562 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:00.920698 25713 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:00.923218 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.975425 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:00.988564 25785 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:00.903184 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.990463 25711 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:00.831077 27424 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.992653 27424 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:00.958942 26342 raft_consensus.cc:1217] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:18:00.994150 27456 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:00.996057 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.998132 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:00.998649 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.999194 25632 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:01.010859 26339 raft_consensus.cc:1217] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:18:00.848857 27439 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:00.851846 27413 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.947160 27412 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:00.965467 27409 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:00.935441 27430 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:00.848062 27433 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.166s
I20260709 12:18:00.853799 27451 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:01.009521 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 168031us
W20260709 12:18:01.142884 25601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.001s	sys 0.011s
W20260709 12:18:01.233104 25601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.005s	sys 0.011s
I20260709 12:18:01.012770 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:01.127612 25710 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:01.059190 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:00.987617 27441 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:01.131394 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:01.233609 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 656738us
I20260709 12:18:01.227707 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:01.232679 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:01.058735 27444 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:18:00.958247 27446 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:01.247450 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:01.010591 27437 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:01.060074 27420 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:01.194731 27445 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:01.001945 27447 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:01.247810 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:01.059155 27453 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:01.250387 25750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.002s	sys 0.014s
I20260709 12:18:01.058085 27436 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:01.195535 27431 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:01.001670 27415 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:01.016567 27448 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:01.059855 27421 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:01.236685 27449 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:01.222424 27410 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:01.059370 27455 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:01.944346 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:01.950655 25750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.360s	user 0.011s	sys 0.019s
W20260709 12:18:01.354765 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.245517 27412 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:01.944550 25783 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:01.279803 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:01.242946 27456 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:01.954295 25784 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:18:01.405859 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:01.368947 27458 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
W20260709 12:18:01.382009 27408 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.858s	user 0.016s	sys 0.004s
W20260709 12:18:01.344630 27428 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.934s	user 0.014s	sys 0.000s
W20260709 12:18:01.958019 27402 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.437s	user 0.022s	sys 0.015s
I20260709 12:18:01.962069 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:01.966665 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:01.354296 27459 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:01.254117 27409 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.706s
I20260709 12:18:01.337822 27404 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:01.953416 27439 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:01.957648 27445 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:01.968288 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:01.952286 27448 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:01.968475 25780 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:01.961684 27455 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:01.011300 27427 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.003001 27455 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:02.004748 27427 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:01.972587 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:01.983322 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:01.958071 27412 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:02.006567 27412 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:02.008534 27459 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:01.986143 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:01.365307 27443 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:01.970892 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:02.011034 25784 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:02.010923 27443 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.011471 27459 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.013447 27443 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:01.304656 27446 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:01.998764 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:02.024258 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.012s	sys 0.041s
I20260709 12:18:01.258751 27430 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:02.024726 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.012s	sys 0.042s
I20260709 12:18:01.969446 27402 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.032076 27391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701d2ef98204ad0bedc9231fe829f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:02.032231 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:02.033061 25778 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:02.034385 25782 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:02.020066 27404 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.034819 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:02.036012 27415 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:02.036144 27466 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:01.966243 27408 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.036424 27404 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:01.999969 27445 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:02.033092 27402 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.037535 25779 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:18:01.961156 27453 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.000793 27448 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:02.020735 27439 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:18:02.041644 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72141us
I20260709 12:18:02.043236 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:01.964149 27436 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.017999 27412 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.037075 27404 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:02.048833 27404 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:02.049418 27403 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.530s	user 0.021s	sys 0.000s
I20260709 12:18:01.970746 27428 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.018028 27446 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:02.040242 27438 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.407s	user 0.008s	sys 0.005s
I20260709 12:18:02.053897 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:02.054525 26338 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:18:02.049862 27403 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.053567 27404 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:02.054275 27438 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.059067 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:02.054662 27408 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.102409 27428 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:02.137986 27412 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.139346 27402 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:02.139889 27402 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.036723 27466 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.044353 27439 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:02.049932 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:02.138211 27455 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.271535 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:02.296284 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:02.301401 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:02.301630 27430 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:02.301714 27460 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:02.229071 25752 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.941s (SENT) [suppressed 6 similar messages]
I20260709 12:18:02.330099 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:02.224020 27439 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:02.332705 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61321.4us
I20260709 12:18:02.226238 27409 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:02.226080 27408 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.227092 27443 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:02.342861 27409 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.347506 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:02.298267 27438 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.328581 27460 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:02.351487 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:02.351634 27468 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:02.226858 27464 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:02.295221 27466 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:02.351650 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:02.361616 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:02.362429 27409 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.354426 27408 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.367304 27453 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:02.376575 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:02.188081 27456 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:01.057878 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:02.377933 25704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 1760) took 2003 ms (client timeout 1941 ms). Trace:
I20260709 12:18:02.378057 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:02.378636 26338 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:02.367776 27443 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:02.332413 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:02.369135 26308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.129s	user 0.005s	sys 0.046s
I20260709 12:18:02.339072 27431 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:02.338828 27404 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:02.378360 25704 rpcz_store.cc:269] 0709 12:18:00.374409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.957846 (+583437us) service_pool.cc:224] Handling call
0709 12:18:02.377845 (+1419999us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.396495 27448 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.396840 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:02.397961 25704 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:02.399049 27448 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:02.387024 27404 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:02.400564 27404 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:02.299872 27402 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.381912 27407 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:02.225276 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:02.383519 26308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.005s	sys 0.046s
I20260709 12:18:02.370620 27460 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:02.352998 27436 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:02.381559 27468 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:02.588358 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 1.34802e+06us
W20260709 12:18:02.651314 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 319055us
I20260709 12:18:02.651641 27455 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:02.387239 27431 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:02.690032 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 460969us
I20260709 12:18:02.375407 27464 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:02.628978 27402 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:02.587848 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 171) took 1423 ms. Trace:
I20260709 12:18:02.816658 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:02.817677 25780 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:02.664685 26335 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:02.753366 27431 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:02.891996 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:02.932878 27445 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.887s	user 0.011s	sys 0.000s
I20260709 12:18:03.170774 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1366) took 1031 ms. Trace:
W20260709 12:18:03.167968 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 564944us
I20260709 12:18:03.171300 25780 rpcz_store.cc:276] 0709 12:18:02.139235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.891499 (+752264us) service_pool.cc:224] Handling call
0709 12:18:03.170682 (+279183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.786294 25704 rpcz_store.cc:276] 0709 12:18:01.173628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.396406 (+1222778us) service_pool.cc:224] Handling call
0709 12:18:02.426857 (+ 30451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:02.831753 27402 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.375016 27402 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:03.376590 27391 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.126s	user 0.006s	sys 0.005s
I20260709 12:18:03.378371 27443 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:01.304207 25567 debug-util.cc:398] Leaking SignalData structure 0x7b080003e460 after lost signal to thread 25570
W20260709 12:18:03.379230 25704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 170) took 2250 ms (client timeout 2242 ms). Trace:
W20260709 12:18:03.379652 25704 rpcz_store.cc:269] 0709 12:18:01.128762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.379004 (+2250242us) service_pool.cc:211] Skipping call since client already timed out
0709 12:18:03.379137 (+   133us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:18:03.378762 27404 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.978s	user 0.006s	sys 0.005s
I20260709 12:18:03.380455 27404 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:03.382180 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.554s	user 0.016s	sys 0.011s
I20260709 12:18:03.380677 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.388588 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.394973 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1360) took 1257 ms. Trace:
I20260709 12:18:03.395442 25780 rpcz_store.cc:276] 0709 12:18:02.137566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.326638 (+1189072us) service_pool.cc:224] Handling call
0709 12:18:03.394887 (+ 68249us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:03.401839 25753 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.003s	sys 0.021s
W20260709 12:18:03.402304 25753 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.003s	sys 0.021s
I20260709 12:18:03.382833 27404 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.324818 27445 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:03.403376 25567 debug-util.cc:398] Leaking SignalData structure 0x7b0800250720 after lost signal to thread 25662
W20260709 12:18:03.415141 25750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.016s	sys 0.016s
W20260709 12:18:03.415652 25750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.016s	sys 0.016s
I20260709 12:18:03.402834 25704 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
W20260709 12:18:03.427009 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.000s	sys 0.035s
W20260709 12:18:03.427572 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.000s	sys 0.035s
W20260709 12:18:03.428773 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.429591 26339 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 1.060s	user 0.039s	sys 0.044s
W20260709 12:18:03.430020 26339 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.039s	sys 0.044s
I20260709 12:18:03.422600 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:03.432231 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.604s	user 0.020s	sys 0.011s
W20260709 12:18:03.432765 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 2.60445e+06us
I20260709 12:18:03.433296 27445 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:03.417557 27460 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.789s	user 0.008s	sys 0.004s
I20260709 12:18:03.437438 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1359) took 1300 ms. Trace:
I20260709 12:18:03.424628 27404 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:03.438100 25780 rpcz_store.cc:276] 0709 12:18:02.137299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.396236 (+1258937us) service_pool.cc:224] Handling call
0709 12:18:03.437383 (+ 41147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.440284 27464 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:03.426364 25704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 1855) took 2228 ms (client timeout 2185 ms). Trace:
W20260709 12:18:03.436561 25567 debug-util.cc:398] Leaking SignalData structure 0x7b080034d500 after lost signal to thread 25737
I20260709 12:18:03.430285 27432 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.443121 27404 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.446971 27479 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:03.447137 27480 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:03.442751 25704 rpcz_store.cc:269] 0709 12:18:01.196463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.380336 (+2183873us) service_pool.cc:224] Handling call
0709 12:18:03.424420 (+ 44084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.454885 27480 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.456176 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1775) took 2353 ms. Trace:
I20260709 12:18:03.456579 25704 rpcz_store.cc:276] 0709 12:18:01.102584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.455318 (+2352734us) service_pool.cc:224] Handling call
0709 12:18:03.455646 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.455830 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.455862 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.455906 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:03.455944 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.456002 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.456121 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.457311 27404 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.444545 27468 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.460345 27479 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.461050 27431 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:03.463311 25567 debug-util.cc:398] Leaking SignalData structure 0x7b08002835a0 after lost signal to thread 25811
W20260709 12:18:03.464516 25567 debug-util.cc:398] Leaking SignalData structure 0x7b080037d880 after lost signal to thread 26368
I20260709 12:18:03.453507 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:03.464352 27479 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:03.465351 27464 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:03.465631 27431 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.466071 27464 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.467747 25780 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:03.451737 27439 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.102s	user 0.008s	sys 0.001s
I20260709 12:18:03.441998 27460 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.471184 27460 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.471493 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 472) took 2875 ms. Trace:
I20260709 12:18:03.471935 26344 rpcz_store.cc:276] 0709 12:18:00.596574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.597086 (+   512us) service_pool.cc:224] Handling call
0709 12:18:00.597515 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.635162 (+ 37647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.635210 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:00.635253 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:00.635297 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.471222 (+2835925us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:03.471270 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.471395 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86091520}
W20260709 12:18:03.508204 25567 debug-util.cc:398] Leaking SignalData structure 0x7b080037d9a0 after lost signal to thread 27383
W20260709 12:18:03.510164 25567 debug-util.cc:398] Leaking SignalData structure 0x7b080036fb60 after lost signal to thread 27391
I20260709 12:18:03.469468 27439 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.513252 27439 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.514520 27464 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:01.362479 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:03.516857 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 1853) took 2496 ms (client timeout 1608 ms). Trace:
W20260709 12:18:03.517201 25706 rpcz_store.cc:269] 0709 12:18:01.020792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.253015 (+232223us) service_pool.cc:224] Handling call
0709 12:18:03.516768 (+2263753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.519171 25780 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:03.519552 27484 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:03.520574 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 172) took 2286 ms. Trace:
I20260709 12:18:03.521289 27483 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:03.521939 27483 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.524703 27483 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.521775 25706 rpcz_store.cc:276] 0709 12:18:01.233752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.517796 (+2284044us) service_pool.cc:224] Handling call
0709 12:18:03.518085 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.518253 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.518275 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.518312 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:03.518338 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.518393 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.520510 (+  2117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.520263 27484 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.526636 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1776) took 2272 ms. Trace:
I20260709 12:18:03.527070 25706 rpcz_store.cc:276] 0709 12:18:01.254464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.525856 (+2271392us) service_pool.cc:224] Handling call
0709 12:18:03.526151 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.526313 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.526337 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.526379 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:03.526408 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.526458 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.526577 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:03.528654 25603 connection.cc:256] RPC call timeout handler was delayed by -1.2953175650000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:18:03.529291 25603 connection.cc:256] RPC call timeout handler was delayed by -1.2738560089999282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 12:18:03.529717 25603 connection.cc:256] RPC call timeout handler was delayed by -1.1595560089999708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260709 12:18:03.527552 27484 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.530985 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 165) took 2582 ms. Trace:
I20260709 12:18:03.531828 25708 rpcz_store.cc:276] 0709 12:18:00.948591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.957982 (+  9391us) service_pool.cc:224] Handling call
0709 12:18:00.995695 (+ 37713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.529809 (+2534114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.529848 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.529877 (+    29us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:03.529899 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.530754 (+   855us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b5400106154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:03.530782 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.530904 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5819209600}
I20260709 12:18:03.534418 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1856) took 2337 ms. Trace:
I20260709 12:18:03.534755 27422 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.534921 25704 rpcz_store.cc:276] 0709 12:18:01.196799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.457642 (+2260843us) service_pool.cc:224] Handling call
0709 12:18:03.457946 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.458149 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:03.509980 (+ 51831us) log.cc:844] Serialized 14 byte log entry
0709 12:18:03.533942 (+ 23962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.534002 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:03.534072 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.534175 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.534353 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60737,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":3141,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":63621,"threads_started":3,"wal-append.queue_time_us":22751}
I20260709 12:18:03.539810 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.540614 27483 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:03.540995 27483 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:03.541369 27483 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:03.541278 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.543246 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.543689 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.545465 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.546119 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:03.546604 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.547003 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.547565 25706 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:03.548370 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1362) took 1410 ms. Trace:
I20260709 12:18:03.548699 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.548745 25781 rpcz_store.cc:276] 0709 12:18:02.138089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.547647 (+1409558us) service_pool.cc:224] Handling call
0709 12:18:03.547913 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.548073 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.548097 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.548137 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:03.548165 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.548216 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.548307 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.551324 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.553079 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.553088 27487 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:03.553862 27487 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.556262 27487 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.553826 25786 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
W20260709 12:18:03.557617 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1361) took 1419 ms (client timeout 1575 ms). Trace:
W20260709 12:18:03.557855 25786 rpcz_store.cc:269] 0709 12:18:02.137830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.552898 (+1415068us) service_pool.cc:224] Handling call
0709 12:18:03.557530 (+  4632us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.560850 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1364) took 1422 ms. Trace:
I20260709 12:18:03.561290 25786 rpcz_store.cc:276] 0709 12:18:02.138663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.558513 (+1419850us) service_pool.cc:224] Handling call
0709 12:18:03.559505 (+   992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.559645 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.559660 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.559690 (+    30us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:03.559737 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.559797 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.560788 (+   991us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.563663 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1367) took 1424 ms. Trace:
I20260709 12:18:03.564070 25786 rpcz_store.cc:276] 0709 12:18:02.139474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.562810 (+1423336us) service_pool.cc:224] Handling call
0709 12:18:03.563133 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.563335 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.563364 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.563407 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:03.563436 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.563492 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.563610 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.566182 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1368) took 1426 ms. Trace:
I20260709 12:18:03.566583 25786 rpcz_store.cc:276] 0709 12:18:02.139713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.565452 (+1425739us) service_pool.cc:224] Handling call
0709 12:18:03.565726 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.565900 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.565923 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.565964 (+    41us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:03.565991 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.566040 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.566133 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.566888 27483 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.568740 25786 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:03.568985 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1363) took 1430 ms. Trace:
I20260709 12:18:03.569453 25781 rpcz_store.cc:276] 0709 12:18:02.138350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.549958 (+1411608us) service_pool.cc:224] Handling call
0709 12:18:03.550219 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.550386 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:03.550407 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:03.550444 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:03.550471 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.568779 (+ 18308us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.568922 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.573765 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.575215 27411 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:03.575675 27411 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:03.576057 27411 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.576732 27483 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.578573 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.580235 27484 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:03.580840 27484 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.583148 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.583328 27484 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.583843 26344 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:18:03.587064 25786 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.584566 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.589252 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.591019 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.592157 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.593102 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.594043 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.596221 25786 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:03.597081 27440 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.600076 27411 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:03.603331 27411 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.603044 27419 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:03.605111 27419 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.607383 27419 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.608074 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.608568 25704 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.612751 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1801) took 2366 ms. Trace:
I20260709 12:18:03.613632 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:03.613219 25785 rpcz_store.cc:276] 0709 12:18:01.246465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.246996 (+   531us) service_pool.cc:224] Handling call
0709 12:18:01.247337 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.956691 (+709354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.956726 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:01.956781 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:01.956808 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.612443 (+1655635us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b5400099554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:03.612500 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.612657 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630886272}
I20260709 12:18:03.614805 27426 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.616307 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.618078 27426 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:03.618579 27426 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:03.619462 27426 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:03.624632 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.625417 25708 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:03.627496 27424 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:03.628183 27424 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:03.628522 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.629145 27424 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.629961 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.631650 27424 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.640393 25706 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.632150 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.633167 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.644397 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.645123 25708 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.637050 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.638062 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.646607 27469 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:03.647034 27469 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:03.647423 27469 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.648108 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.648253 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.648448 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:03.648967 26339 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:03.649277 26338 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:03.649451 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.650460 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.638414 27440 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:03.659430 25706 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:03.660676 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.661784 25706 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:03.663908 27440 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.666931 27440 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.667423 27440 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:03.664490 27411 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:03.668843 27411 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.671696 27411 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.673336 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1843) took 2910 ms. Trace:
I20260709 12:18:03.673755 25713 rpcz_store.cc:276] 0709 12:18:00.762980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.765348 (+  2368us) service_pool.cc:224] Handling call
0709 12:18:00.765746 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.987764 (+222018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.673070 (+2685306us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54003ac854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:03.673264 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355264896}
I20260709 12:18:03.653952 26340 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:03.674721 25706 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:03.675997 25752 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.677563 27472 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:03.676836 27484 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:03.678362 27472 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:03.678481 27484 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.680672 26335 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:03.680860 27484 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.678599 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.682005 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.682706 25713 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.685310 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:03.685923 26338 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:03.686870 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.687300 27483 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:03.687692 26335 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:03.687834 27483 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.688867 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.695000 25713 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:03.696525 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.667919 27440 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:03.698432 25710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 1854) took 2502 ms (client timeout 2053 ms). Trace:
I20260709 12:18:03.699939 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:03.703197 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:03.705242 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.705433 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:03.705029 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 83685.8us
I20260709 12:18:03.705922 25713 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.706291 25785 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:03.662832 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.705729 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:03.700306 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.699800 27411 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:03.712453 25750 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:03.713114 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.711380 27405 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:03.704360 27488 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:03.718829 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.704120 27426 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:03.713524 25704 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:03.715677 27440 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.716123 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.699862 27450 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:18:03.726643 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.727248 27423 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:03.700790 25710 rpcz_store.cc:269] 0709 12:18:01.196114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.253344 (+ 57230us) service_pool.cc:224] Handling call
0709 12:18:03.688659 (+2435315us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5800004b98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:03.698335 (+  9676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250491776}
I20260709 12:18:03.720192 27405 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:03.729038 25713 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:03.716187 27411 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.729341 27432 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:03.730584 27405 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.730962 27432 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.731940 27411 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.732379 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.733197 27432 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.733654 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.720824 27484 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.734995 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.719408 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:03.679615 27472 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.736182 27483 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.728571 27426 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.716709 27443 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 12:18:03.725984 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1358) took 1589 ms (client timeout 1801 ms). Trace:
I20260709 12:18:03.727882 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.720474 27479 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.737130 27478 raft_consensus.cc:1064] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
W20260709 12:18:03.737062 25783 rpcz_store.cc:269] 0709 12:18:02.136991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.715893 (+1578902us) service_pool.cc:224] Handling call
0709 12:18:03.725929 (+ 10036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.729514 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.737783 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:03.741324 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:03.727283 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.744400 27426 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.744716 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.745831 25781 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:03.746801 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.747424 25704 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:03.748342 27456 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:03.748734 27443 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:03.749027 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.749218 27443 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:03.749647 25713 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:03.748380 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.746392 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:03.751055 25709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 1773) took 2759 ms (client timeout 2178 ms). Trace:
W20260709 12:18:03.751428 25709 rpcz_store.cc:269] 0709 12:18:00.991731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.991895 (+   164us) service_pool.cc:224] Handling call
0709 12:18:03.746223 (+2754328us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b5800004b98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:03.750981 (+  4758us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8440192}
I20260709 12:18:03.751832 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:03.728291 27488 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:03.749614 27443 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.756657 27464 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:03.758081 25603 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.760856 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.758991 25603 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:03.762112 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1353) took 1656 ms. Trace:
I20260709 12:18:03.762842 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.764149 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.765295 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.766988 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.768404 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.762668 25784 rpcz_store.cc:276] 0709 12:18:02.101205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.102939 (+  1734us) service_pool.cc:224] Handling call
0709 12:18:02.301381 (+198442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.410259 (+108878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:02.410306 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:02.410367 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:02.410411 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.757622 (+1347211us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400098b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:03.757657 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.757760 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249668480}
I20260709 12:18:03.757071 27464 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:03.748939 27456 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:03.772073 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.772535 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.756393 27460 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:03.773262 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.773612 27491 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:03.775846 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.769886 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.776029 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.748773 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.789615 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.789674 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.790146 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:03.778012 25601 connection.cc:256] RPC call timeout handler was delayed by -1.1627593230000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 12:18:03.782101 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.782019 27464 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.762154 27440 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:03.793555 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.793784 27440 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:03.782869 27478 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.778729 27460 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:03.778594 27451 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.785449 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.766745 27483 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:03.797336 27483 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:03.797343 26341 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:18:03.798094 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 373) took 1452 ms. Trace:
I20260709 12:18:03.787037 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.757453 27426 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:03.742746 25708 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:03.778542 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:03.790556 27491 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:03.828886 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.830463 25708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 174) took 2487 ms (client timeout 1607 ms). Trace:
W20260709 12:18:03.830914 25708 rpcz_store.cc:269] 0709 12:18:01.343525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.830270 (+2486745us) service_pool.cc:211] Skipping call since client already timed out
0709 12:18:03.830400 (+   130us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:18:03.831909 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.833016 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:03.833500 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 1804) took 2554 ms (client timeout 1964 ms). Trace:
W20260709 12:18:03.833863 25779 rpcz_store.cc:269] 0709 12:18:01.278818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.279414 (+   596us) service_pool.cc:224] Handling call
0709 12:18:03.833327 (+2553913us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b54000b09d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:03.833420 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182619520}
I20260709 12:18:03.834820 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.835774 27488 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:03.838085 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:03.839150 26337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 545) took 1920 ms (client timeout 1886 ms). Trace:
W20260709 12:18:03.840601 26337 rpcz_store.cc:269] 0709 12:18:01.919181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.955906 (+ 36725us) service_pool.cc:224] Handling call
0709 12:18:03.837794 (+1881888us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:03.839051 (+  1257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5239552}
I20260709 12:18:03.790354 26339 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:18:03.792568 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.789358 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:03.785058 27456 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.795042 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.794236 26343 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:03.896623 26344 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 1.
I20260709 12:18:03.798411 26341 rpcz_store.cc:276] 0709 12:18:02.345758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.785198 (+1439440us) service_pool.cc:224] Handling call
0709 12:18:03.798037 (+ 12839us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.802053 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:03.805558 27460 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.812750 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 536) took 2823 ms. Trace:
I20260709 12:18:03.918440 26342 rpcz_store.cc:276] 0709 12:18:00.997875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.008714 (+ 10839us) service_pool.cc:224] Handling call
0709 12:18:01.127297 (+118583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.958696 (+831399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.958726 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:01.958766 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:01.958800 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.812433 (+1853633us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54004d6554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:03.812506 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.812675 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911598976}
I20260709 12:18:03.844103 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1802) took 2588 ms. Trace:
I20260709 12:18:04.340732 25782 rpcz_store.cc:276] 0709 12:18:01.256132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.256458 (+   326us) service_pool.cc:224] Handling call
0709 12:18:01.256845 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.034937 (+778092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.843715 (+1808778us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54001a63d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:03.844010 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787639168}
W20260709 12:18:03.790782 25601 connection.cc:256] RPC call timeout handler was delayed by -1.0691593229998944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
I20260709 12:18:03.879470 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:03.988098 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:04.343143 25601 connection.cc:256] RPC call timeout handler was delayed by -1.0682593229998929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
I20260709 12:18:04.343916 25629 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:04.344772 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.344969 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.345754 25638 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:04.345921 25629 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:03.989579 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.347143 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.348304 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.349643 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.009505 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.351783 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:04.352662 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.353257 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:03.927129 25708 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:03.934747 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.356069 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.356197 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.357391 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.357551 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:01.338102 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.354240 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:04.359464 25712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 1857) took 3102 ms (client timeout 1503 ms). Trace:
I20260709 12:18:04.359617 25704 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
W20260709 12:18:04.359917 25712 rpcz_store.cc:269] 0709 12:18:01.256572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.330292 (+ 73720us) service_pool.cc:224] Handling call
0709 12:18:04.359172 (+3028880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:04.350337 25635 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.924471 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:04.430933 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:04.432261 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:04.433014 26339 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:04.436982 25533 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:03.814042 27391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56168 (request call id 469) took 5913 ms. Trace:
I20260709 12:18:04.447600 27391 rpcz_store.cc:276] 0709 12:17:57.900484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.900638 (+   154us) service_pool.cc:224] Handling call
0709 12:17:57.901792 (+  1154us) ts_tablet_manager.cc:934] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
0709 12:18:01.962744 (+4060952us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:02.249820 (+287076us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:02.249973 (+   153us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:03.377200 (+1127227us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:03.377553 (+   353us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:03.813894 (+436341us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:03.813960 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":263,"tablet-copy.queue_time_us":257,"thread_start_us":2205971,"threads_started":1}
I20260709 12:18:03.817111 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.919979 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:03.935906 27469 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:03.818030 27464 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.916472 27403 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:03.816138 27474 ts_tablet_manager.cc:933] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:18:03.927714 27405 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.828191 27491 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.934654 27479 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:03.799801 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:03.794272 27440 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.797883 27483 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.881489 25785 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
W20260709 12:18:03.818969 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.916496 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:04.532814 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60989us
I20260709 12:18:04.533216 26340 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 2: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:04.532825 27491 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:03.924326 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:03.856930 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:04.030859 25677 connection.cc:256] RPC call timeout handler was delayed by -1.9746252320001076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 12:18:04.541118 25677 connection.cc:256] RPC call timeout handler was delayed by -1.9593252320000829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:18:04.541570 25677 connection.cc:256] RPC call timeout handler was delayed by -1.8108252320000702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 12:18:04.541659 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1356) took 2439 ms (client timeout 2000 ms). Trace:
W20260709 12:18:04.541903 25677 connection.cc:256] RPC call timeout handler was delayed by -1.6794252320000851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 12:18:04.048790 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.055440 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.045977 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.042155 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1753) took 3669 ms. Trace:
W20260709 12:18:04.058368 25676 connection.cc:256] RPC call timeout handler was delayed by -1.9820981559998927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
I20260709 12:18:04.117600 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.120609 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.138589 27408 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:04.169683 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:04.177342 27406 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:04.225492 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:04.542246 25677 connection.cc:256] RPC call timeout handler was delayed by -1.5795252320001509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:18:04.229234 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 75743.4us
I20260709 12:18:04.230997 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.232064 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.241528 27443 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:04.247051 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1348) took 2204 ms. Trace:
I20260709 12:18:01.314904 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.288674 27480 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:01.407981 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.303781 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.319377 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 483) took 3032 ms. Trace:
W20260709 12:18:04.329485 26338 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.643s	user 0.023s	sys 0.000s
I20260709 12:18:04.331734 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.336236 27435 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:04.363209 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.599973 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.601548 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.366674 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.603338 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.603719 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.604857 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.365764 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.607741 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.585382 25778 rpcz_store.cc:276] 0709 12:18:02.042434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.044116 (+  1682us) service_pool.cc:224] Handling call
0709 12:18:04.244973 (+2200857us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:04.245388 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.246112 (+   724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.246147 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:04.246235 (+    88us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:04.246289 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.246422 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:04.246621 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54908672}
I20260709 12:18:04.604964 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.609056 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.609830 25638 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:04.366973 27453 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:04.610841 27453 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:04.611456 27453 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:04.613401 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:04.367949 27457 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:04.368139 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.365623 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.377683 25635 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
W20260709 12:18:04.399466 25601 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:60736 dropped due to backpressure: service queue is full with 50 items
W20260709 12:18:04.404699 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.412081 25604 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:60736 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1672)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1664)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1653)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1674)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1382)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1671)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1676)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 310)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1651)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1649)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1670)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1663)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1654)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1673)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1669)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1678)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 306)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 317)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 318)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1646)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1656)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1657)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1659)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1675)
I20260709 12:18:04.422003 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:04.460327 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.461120 25786 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:04.462133 27469 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:04.450239 27391 tablet_copy_client.cc:806] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:18:04.625916 27391 tablet_copy_client.cc:670] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:18:04.618805 27457 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:04.636494 27457 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:04.462875 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:04.475142 27474 tablet_copy_client.cc:323] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
W20260709 12:18:04.503340 27460 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.012s	sys 0.001s
I20260709 12:18:04.525713 27405 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:04.227106 26337 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
W20260709 12:18:04.583173 25676 connection.cc:256] RPC call timeout handler was delayed by -1.8983175729999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 12:18:04.583400 25677 connection.cc:256] RPC call timeout handler was delayed by -1.5768252320001466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
I20260709 12:18:04.586059 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.582459 25705 rpcz_store.cc:276] 0709 12:18:00.372392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.627988 (+255596us) service_pool.cc:224] Handling call
0709 12:18:00.628334 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.772906 (+144572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.772942 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:00.772993 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:00.773037 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.041353 (+3268316us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000730d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:04.041465 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:04.041849 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331975936}
I20260709 12:18:04.587038 26336 rpcz_store.cc:276] 0709 12:18:01.287211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.287535 (+   324us) service_pool.cc:224] Handling call
0709 12:18:01.287870 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.016715 (+728845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:02.016760 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:02.016819 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:02.016862 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.306173 (+2289311us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b54005a0fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:04.306268 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:04.319131 (+ 12863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1675507968}
W20260709 12:18:04.643520 25677 connection.cc:256] RPC call timeout handler was delayed by -1.5480252320001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
I20260709 12:18:04.588893 25636 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:18:04.589074 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.589426 27443 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:04.643066 25676 connection.cc:256] RPC call timeout handler was delayed by -1.883017572999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:18:04.645619 25676 connection.cc:256] RPC call timeout handler was delayed by -1.7345175729999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 12:18:04.646077 25676 connection.cc:256] RPC call timeout handler was delayed by -1.6796175730000868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 12:18:04.646476 25676 connection.cc:256] RPC call timeout handler was delayed by -1.6796175730000868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 12:18:04.646888 25676 connection.cc:256] RPC call timeout handler was delayed by -1.6031175729999632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 12:18:04.647261 25676 connection.cc:256] RPC call timeout handler was delayed by -1.5194175730000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
W20260709 12:18:04.647646 25676 connection.cc:256] RPC call timeout handler was delayed by -1.5194175730000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
W20260709 12:18:04.648012 25676 connection.cc:256] RPC call timeout handler was delayed by -1.503217573000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:18:04.648437 25676 connection.cc:256] RPC call timeout handler was delayed by -1.5005175730000246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
W20260709 12:18:04.648808 25676 connection.cc:256] RPC call timeout handler was delayed by -1.4483175729999402s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:18:04.649183 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3439175729999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260709 12:18:04.649572 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3241175729999668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 12:18:04.649941 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3196175729999595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
W20260709 12:18:04.589581 25707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 173) took 3335 ms (client timeout 1723 ms). Trace:
I20260709 12:18:04.590226 25630 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:04.590907 25634 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:04.590986 25632 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:04.594616 26338 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:04.597182 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:04.641127 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:60736 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.653162 25711 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:04.597563 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.588021 25631 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:04.601245 27437 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:04.610142 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.369406 27477 raft_consensus.cc:1064] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
W20260709 12:18:04.542069 25787 rpcz_store.cc:269] 0709 12:18:02.102163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.102846 (+   683us) service_pool.cc:224] Handling call
0709 12:18:03.856555 (+1753709us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:04.541402 (+684847us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5433600}
I20260709 12:18:04.620076 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.624044 25712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:34626
W20260709 12:18:04.643949 25677 connection.cc:256] RPC call timeout handler was delayed by -1.5246252320000622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:18:04.661029 25677 connection.cc:256] RPC call timeout handler was delayed by -1.4202252320001207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260709 12:18:04.661515 25677 connection.cc:256] RPC call timeout handler was delayed by -1.4004252320000887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 12:18:04.661913 25677 connection.cc:256] RPC call timeout handler was delayed by -1.3959252320000815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
W20260709 12:18:04.650326 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3034175729999333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
W20260709 12:18:04.662328 25677 connection.cc:256] RPC call timeout handler was delayed by -1.3644252320000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 12:18:04.662957 25677 connection.cc:256] RPC call timeout handler was delayed by -1.299154678999912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
W20260709 12:18:04.663313 25677 connection.cc:256] RPC call timeout handler was delayed by -1.2818855890000123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 12:18:04.662437 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3034175729999333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 12:18:04.663712 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:04.664125 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2881175729999086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 12:18:04.650763 25707 rpcz_store.cc:269] 0709 12:18:01.254406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.365228 (+110822us) service_pool.cc:224] Handling call
0709 12:18:04.589489 (+3224261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:04.664688 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2637251649998689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 12:18:04.665238 25676 connection.cc:256] RPC call timeout handler was delayed by -1.1734953180000502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 12:18:04.663720 25677 connection.cc:256] RPC call timeout handler was delayed by -1.22340059499993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
W20260709 12:18:04.664307 25636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1370) took 3341 ms (client timeout 2236 ms). Trace:
I20260709 12:18:04.667425 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.660521 25712 raft_consensus.cc:606] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
W20260709 12:18:04.666227 25636 rpcz_store.cc:269] 0709 12:18:01.314331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.314517 (+   186us) service_pool.cc:224] Handling call
0709 12:18:04.644080 (+3329563us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:04.663893 25632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1371) took 3252 ms (client timeout 2201 ms). Trace:
I20260709 12:18:04.672386 25637 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.672513 26341 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 2: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:04.669865 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:04.671456 25632 rpcz_store.cc:269] 0709 12:18:01.406975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.407625 (+   650us) service_pool.cc:224] Handling call
0709 12:18:04.652278 (+3244653us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:04.658635 27460 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.013s	sys 0.001s
I20260709 12:18:04.667361 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.677692 25708 raft_consensus.cc:2376] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 3. Re-sending same reply.
I20260709 12:18:04.682725 25711 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:04.683574 27491 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:04.658915 27469 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:04.684823 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:04.692893 25780 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:04.693071 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:04.700372 27440 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:04.690743 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 498) took 1007 ms. Trace:
I20260709 12:18:04.701951 25629 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:04.702733 25786 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:04.702910 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:04.701601 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:04.709443 25712 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:04.709942 25706 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:04.703403 26338 rpcz_store.cc:276] 0709 12:18:03.683156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.685028 (+  1872us) service_pool.cc:224] Handling call
0709 12:18:04.680620 (+995592us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:04.691205 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 66126.1us
I20260709 12:18:04.722047 25712 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:04.726701 27453 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:04.619545 25601 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:60736 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1653)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1674)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1382)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1650)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1671)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1676)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 310)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1651)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1667)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1665)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1649)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1670)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1663)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1654)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1673)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1658)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1669)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1678)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 306)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 317)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 318)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1646)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1647)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1648)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1655)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1656)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1657)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1659)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1675)
I20260709 12:18:04.728879 27453 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:04.709466 27477 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:04.715389 27469 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:04.733021 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:04.733999 27469 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:04.735030 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.735877 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.736671 25630 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:04.739408 25634 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:04.733845 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.735940 25636 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.754988 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.698144 27483 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:04.760538 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.758453 25713 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:18:04.762002 25710 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:04.757548 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.764394 27447 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:04.765470 25706 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:04.765568 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.768662 25630 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.769302 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.769812 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.770028 25631 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.771055 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.771739 25631 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:04.772775 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1823) took 1005 ms. Trace:
I20260709 12:18:04.766618 27500 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:04.767409 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.779402 25629 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:04.767797 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:04.775529 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet ef4fd24cfa184aa8a8a1de8f32296e6a from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:18:04.780537 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet ef4fd24cfa184aa8a8a1de8f32296e6a from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-ef4fd24cfa184aa8a8a1de8f32296e6a
I20260709 12:18:04.776439 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.783090 25638 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:04.784276 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.784454 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.785223 25631 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:04.779191 27500 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:04.787370 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.803260 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.803864 25638 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.804673 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.805939 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.789535 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.807123 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.807825 25638 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:04.808167 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.808815 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.809480 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.773243 25781 rpcz_store.cc:276] 0709 12:18:03.767347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.768147 (+   800us) service_pool.cc:224] Handling call
0709 12:18:04.498872 (+730725us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:04.499642 (+   770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.499681 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:04.500327 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 12:18:04.618980 (+118653us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:04.619029 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.639647 (+ 20618us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:04.772423 (+132776us) raft_consensus.cc:1685] finished
0709 12:18:04.772442 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:04.772586 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":701,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":203,"thread_start_us":106371,"threads_started":1,"wal-append.queue_time_us":223472}
I20260709 12:18:04.809955 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.811010 27440 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:04.799220 25634 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:04.801008 27460 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:04.812461 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.787863 27483 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:04.786105 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.795632 27500 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:04.814026 25634 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:18:04.814370 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.815243 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.816000 25634 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.801762 25629 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:04.817147 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:04.818717 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:04.820268 27460 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:04.820938 27460 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:04.822623 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 43757.1us
I20260709 12:18:04.823484 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.822222 27460 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:04.825656 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.827247 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.825057 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.815855 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.801400 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:04.814522 27500 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:04.837733 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.840011 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.834754 27435 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:04.833817 27503 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:18:04.810279 25636 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:04.858115 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.858980 25634 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.848969 25630 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
W20260709 12:18:04.892406 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.962167 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:04.962510 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 107093us
W20260709 12:18:04.962098 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 170008us
I20260709 12:18:04.905534 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.972997 25631 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:04.982446 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:04.983366 25633 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:04.846892 27488 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:18:05.240594 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.240450 26338 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:05.255985 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:05.256666 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:05.239009 25734 debug-util.cc:398] Leaking SignalData structure 0x7b08000f7340 after lost signal to thread 24424
W20260709 12:18:05.258793 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 457469us
W20260709 12:18:05.035159 26309 leader_election.cc:336] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.607s (SENT)
I20260709 12:18:05.257030 25706 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:05.261998 25712 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:05.253926 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:05.126399 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.165802 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 73815.2us
I20260709 12:18:04.890167 27469 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:05.263038 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:04.854054 27447 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:05.259567 25734 debug-util.cc:398] Leaking SignalData structure 0x7b08000f7f00 after lost signal to thread 25570
I20260709 12:18:05.275513 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:05.291760 27447 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:05.308259 25734 debug-util.cc:398] Leaking SignalData structure 0x7b0800345c00 after lost signal to thread 25662
I20260709 12:18:05.309298 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:05.314313 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:05.315070 26344 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:05.315842 26341 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:18:05.319902 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:05.321256 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 313) took 1245 ms. Trace:
I20260709 12:18:05.321784 25632 rpcz_store.cc:276] 0709 12:18:04.075607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.319442 (+1243835us) service_pool.cc:224] Handling call
0709 12:18:05.321172 (+  1730us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.332240 25632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:18:05.333001 25632 raft_consensus.cc:606] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:05.327226 27477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.050s	sys 0.018s
W20260709 12:18:05.335141 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50650us
W20260709 12:18:05.310322 25734 debug-util.cc:398] Leaking SignalData structure 0x7b080034f0e0 after lost signal to thread 25737
I20260709 12:18:05.354115 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 297) took 1279 ms. Trace:
I20260709 12:18:05.355397 26344 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
W20260709 12:18:05.366725 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.354897 25637 rpcz_store.cc:276] 0709 12:18:04.067681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.330398 (+1262717us) service_pool.cc:224] Handling call
0709 12:18:05.330812 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.337790 (+  6978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.337827 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.345540 (+  7713us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.345597 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.347010 (+  1413us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.347184 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:05.337600 27477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.050s	sys 0.018s
W20260709 12:18:05.374837 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 339750us
W20260709 12:18:05.376566 25734 debug-util.cc:398] Leaking SignalData structure 0x7b080039d140 after lost signal to thread 25811
W20260709 12:18:05.378255 25734 debug-util.cc:398] Leaking SignalData structure 0x7b080039da20 after lost signal to thread 26368
I20260709 12:18:05.379963 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 301) took 1310 ms. Trace:
I20260709 12:18:05.380599 25632 rpcz_store.cc:276] 0709 12:18:04.069091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.378736 (+1309645us) service_pool.cc:224] Handling call
0709 12:18:05.379240 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.379466 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.379506 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.379595 (+    89us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.379639 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.379717 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.379888 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.383275 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 299) took 1314 ms. Trace:
I20260709 12:18:05.383805 25637 rpcz_store.cc:276] 0709 12:18:04.068421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.369923 (+1301502us) service_pool.cc:224] Handling call
0709 12:18:05.370363 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.382757 (+ 12394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.382801 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.382868 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.382911 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.383007 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.383175 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.386054 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 307) took 1312 ms. Trace:
W20260709 12:18:05.388208 27457 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.751s	user 0.012s	sys 0.000s
I20260709 12:18:05.388839 27457 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:05.391991 25637 rpcz_store.cc:276] 0709 12:18:04.073253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.384987 (+1311734us) service_pool.cc:224] Handling call
0709 12:18:05.385407 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.385658 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.385696 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.385748 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.385788 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.385855 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.385991 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.395556 25632 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:05.396611 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 306) took 1323 ms. Trace:
W20260709 12:18:05.398523 26309 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.718s (SENT)
I20260709 12:18:05.397122 25632 rpcz_store.cc:276] 0709 12:18:04.072928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.381816 (+1308888us) service_pool.cc:224] Handling call
0709 12:18:05.382263 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.396223 (+ 13960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.396537 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.414650 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 317) took 1337 ms. Trace:
I20260709 12:18:05.415162 25632 rpcz_store.cc:276] 0709 12:18:04.077017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.399793 (+1322776us) service_pool.cc:224] Handling call
0709 12:18:05.413302 (+ 13509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.414192 (+   890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.414234 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.414297 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.414336 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.414409 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.414581 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:05.418697 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 323724us
I20260709 12:18:05.418799 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 311) took 1324 ms. Trace:
I20260709 12:18:05.419718 25637 rpcz_store.cc:276] 0709 12:18:04.074885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.393306 (+1318421us) service_pool.cc:224] Handling call
0709 12:18:05.393684 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.393882 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.393918 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.394025 (+   107us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.394069 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.394135 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.399310 (+  5175us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:05.437237 26309 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.723s (SENT)
I20260709 12:18:05.441963 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:05.442582 25632 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:05.444811 25637 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:05.445500 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1646) took 1101 ms. Trace:
I20260709 12:18:05.445783 25637 rpcz_store.cc:276] 0709 12:18:04.344378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.439767 (+1095389us) service_pool.cc:224] Handling call
0709 12:18:05.444091 (+  4324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.445233 (+  1142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.445442 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.446950 25637 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:05.450691 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1647) took 1106 ms. Trace:
I20260709 12:18:05.451210 25706 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:05.451617 25637 rpcz_store.cc:276] 0709 12:18:04.344751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.446511 (+1101760us) service_pool.cc:224] Handling call
0709 12:18:05.446790 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.450267 (+  3477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.450635 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.454028 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 318) took 1372 ms. Trace:
I20260709 12:18:05.454548 25632 rpcz_store.cc:276] 0709 12:18:04.077349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.416195 (+1338846us) service_pool.cc:224] Handling call
0709 12:18:05.441619 (+ 25424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.448942 (+  7323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.449986 (+  1044us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:05.455892 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 604786us
W20260709 12:18:05.456552 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 196904us
W20260709 12:18:05.523473 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55886.2us
W20260709 12:18:05.525315 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 69687.7us
I20260709 12:18:05.533517 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1648) took 1187 ms. Trace:
I20260709 12:18:05.534010 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1655) took 1149 ms. Trace:
I20260709 12:18:05.534039 25637 rpcz_store.cc:276] 0709 12:18:04.345102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.453334 (+1108232us) service_pool.cc:224] Handling call
0709 12:18:05.466960 (+ 13626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.526897 (+ 59937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.526938 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.529236 (+  2298us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:05.532644 (+  3408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.532720 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.532831 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.534950 25632 rpcz_store.cc:276] 0709 12:18:04.382233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.455484 (+1073251us) service_pool.cc:224] Handling call
0709 12:18:05.524061 (+ 68577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.526974 (+  2913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.527006 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.529141 (+  2135us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.531164 (+  2023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.531257 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.531421 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:05.533047 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 184927us
I20260709 12:18:05.539244 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:05.535633 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 160157us
I20260709 12:18:05.552107 25637 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:05.531615 27457 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:05.557479 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1656) took 1174 ms. Trace:
I20260709 12:18:05.558624 25637 rpcz_store.cc:276] 0709 12:18:04.382635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.536201 (+1153566us) service_pool.cc:224] Handling call
0709 12:18:05.551877 (+ 15676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.555705 (+  3828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.557415 (+  1710us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:05.562345 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.515s (SENT)
W20260709 12:18:05.563360 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.533s (SENT)
I20260709 12:18:05.563558 27513 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:05.563848 27512 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:05.564124 27513 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:05.564543 27513 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:05.564424 27512 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:05.564620 25751 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.535s (SENT)
I20260709 12:18:05.560081 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1659) took 1176 ms. Trace:
I20260709 12:18:05.567893 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:05.568514 25638 rpcz_store.cc:276] 0709 12:18:04.383688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.555981 (+1172293us) service_pool.cc:224] Handling call
0709 12:18:05.556284 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.559676 (+  3392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.559706 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.559750 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:05.559795 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.559850 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.560025 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.584097 27457 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:05.586448 26309 leader_election.cc:336] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.804s (SENT)
I20260709 12:18:05.587002 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:05.588863 26309 leader_election.cc:336] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.888s (SENT)
I20260709 12:18:05.590121 27457 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:05.590637 27457 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:05.590662 27512 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:05.591144 27457 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:05.592218 27460 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:05.592227 27515 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.593266 26339 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:05.593380 27515 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:05.594595 27518 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:05.594821 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1661) took 1210 ms. Trace:
I20260709 12:18:05.595245 25637 rpcz_store.cc:276] 0709 12:18:04.384333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.559350 (+1175017us) service_pool.cc:224] Handling call
0709 12:18:05.575118 (+ 15768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.594419 (+ 19301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.594457 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.594528 (+    71us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:05.594575 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.594645 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.594761 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.595944 27515 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:05.597216 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1675) took 1200 ms. Trace:
I20260709 12:18:05.597604 27477 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:05.593261 26344 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:18:05.599102 27519 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:18:05.599931 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 159009us
I20260709 12:18:05.600495 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1657) took 1217 ms. Trace:
I20260709 12:18:05.597610 25637 rpcz_store.cc:276] 0709 12:18:04.396954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.596286 (+1199332us) service_pool.cc:224] Handling call
0709 12:18:05.596604 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.596858 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.596886 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.596945 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:05.596980 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.597033 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.597130 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.602226 25632 rpcz_store.cc:276] 0709 12:18:04.383071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.553930 (+1170859us) service_pool.cc:224] Handling call
0709 12:18:05.568275 (+ 14345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.600177 (+ 31902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.600208 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.600253 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:05.600290 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.600345 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.600442 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.601603 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:05.604911 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 72592.3us
I20260709 12:18:05.605338 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:05.601080 27519 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:05.614650 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:05.616611 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52036.8us
I20260709 12:18:05.617766 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1668) took 1231 ms. Trace:
I20260709 12:18:05.618100 25638 rpcz_store.cc:276] 0709 12:18:04.386465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.576592 (+1190127us) service_pool.cc:224] Handling call
0709 12:18:05.601848 (+ 25256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.606045 (+  4197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.614388 (+  8343us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.617582 (+  3194us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:05.617616 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.617655 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.617727 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:05.615985 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 89690.1us
W20260709 12:18:05.619237 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 57187.5us
W20260709 12:18:05.619954 25751 leader_election.cc:336] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.598s (SENT)
I20260709 12:18:05.621304 25632 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:05.622920 25637 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:05.623013 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:05.626004 26344 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:05.615327 27521 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:18:05.628499 26309 leader_election.cc:336] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.103s (SENT)
W20260709 12:18:05.650290 27500 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.003s
I20260709 12:18:05.626607 27515 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 3 to 6, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:05.655237 27519 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:05.656422 27519 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:05.644759 27513 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:05.666261 27513 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:05.667755 27523 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:05.668582 27518 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:05.677522 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:05.686594 27521 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:05.690001 27521 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:05.691545 27515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50958 (request call id 91) took 6863 ms. Trace:
I20260709 12:18:05.692623 27515 rpcz_store.cc:276] 0709 12:17:58.828288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.526792 (+1698504us) service_pool.cc:224] Handling call
0709 12:18:00.545661 (+ 18869us) log.cc:844] Serialized 501 byte log entry
0709 12:18:03.735160 (+3189499us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b540039a420. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5904dbe 00007f29b590f930 00007f29b590f8d8 00007f29b590f858 00007f29b590f813 00007f29b590ea9c 00007f29bb7fa2f6 00007f29bb7f4aa8 00007f29b594b7cb 00007f29b594e2c0
0709 12:18:05.691407 (+1956247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":316387,"raft.run_cpu_time_us":3430,"raft.run_wall_time_us":336088,"spinlock_wait_cycles":1727274240,"thread_start_us":237229,"threads_started":2,"wal-append.queue_time_us":164242,"wal-append.run_cpu_time_us":48665,"wal-append.run_wall_time_us":4025738}
W20260709 12:18:05.768699 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 148215us
W20260709 12:18:05.774331 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 152466us
I20260709 12:18:05.775998 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:05.777217 27529 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:05.779704 27529 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:05.782825 27529 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:05.781939 26309 leader_election.cc:336] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.183s (SENT)
I20260709 12:18:05.785122 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:05.786139 27521 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:05.786875 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.185s (SENT)
W20260709 12:18:05.788244 26309 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.205s (SENT)
I20260709 12:18:05.788693 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:05.784157 27529 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:05.797775 25750 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:05.798094 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 178159us
W20260709 12:18:05.798934 25751 leader_election.cc:336] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.598s (SENT)
I20260709 12:18:05.799853 27527 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:05.802695 27523 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:05.803143 27523 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.803295 27521 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:05.803520 27523 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:05.804941 26309 leader_election.cc:336] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.242s (SENT)
W20260709 12:18:05.884014 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 84232.4us
I20260709 12:18:04.986239 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:05.933926 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1387) took 1084 ms. Trace:
I20260709 12:18:05.934319 25630 rpcz_store.cc:276] 0709 12:18:04.849230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.985821 (+136591us) service_pool.cc:224] Handling call
0709 12:18:05.933833 (+948012us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.802287 27479 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
W20260709 12:18:05.943583 25751 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.683s (SENT)
I20260709 12:18:05.951179 27523 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:05.955075 27433 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:05.957367 27433 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:05.959077 27433 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:05.960984 27433 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:05.961611 27506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.007s	sys 0.014s
W20260709 12:18:05.962044 27506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.007s	sys 0.014s
W20260709 12:18:05.968803 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 349709us
I20260709 12:18:05.955224 27523 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:05.953130 25751 leader_election.cc:336] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.655s (SENT)
I20260709 12:18:05.970160 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:05.971644 25751 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.809s (SENT)
W20260709 12:18:05.974669 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.975279 27529 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:05.976400 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 178748us
W20260709 12:18:05.978658 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.987166 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:05.993114 27479 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:05.993682 27479 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:05.994714 27479 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:06.013335 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.014885 25635 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:06.021463 25629 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:06.021890 25629 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:06.022766 25629 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:06.024425 25648 tablet_copy_source_session.cc:215] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:18:06.026032 25648 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:60736 (request call id 320) took 1288 ms. Trace:
I20260709 12:18:06.026471 25648 rpcz_store.cc:276] 0709 12:18:04.759541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.775073 (+ 15532us) service_pool.cc:224] Handling call
0709 12:18:06.025917 (+1250844us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:06.033674 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.035907 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.037182 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.037473 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.038250 25631 raft_consensus.cc:2376] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 3. Re-sending same reply.
I20260709 12:18:06.038827 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.039557 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.040002 25632 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:06.040323 25637 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:06.047236 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.047902 25637 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:06.049299 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.055258 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.056401 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.057430 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.058399 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:06.058772 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 66005.8us
I20260709 12:18:06.058926 25630 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:06.059759 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.060269 25630 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:06.066747 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.067991 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.069597 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.073683 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.074304 25637 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:06.077035 25635 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:06.075812 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:06.078810 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.079598 25637 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:06.079761 27517 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:06.080288 27517 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:06.080796 27517 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.081094 27529 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:06.083448 25632 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:18:06.085810 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.087652 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.089331 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.089744 27511 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:06.090524 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:06.090236 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.092708 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.093194 27511 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:06.093293 25632 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:06.093604 27511 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:06.094696 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.094740 27511 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:06.096266 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 595303b3057546cbbfda4b84f8df8912 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 8)
I20260709 12:18:06.096632 27511 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:06.097332 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.100509 27515 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:06.102880 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:06.104149 27517 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:06.104569 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 1.2469e+06us
I20260709 12:18:06.105273 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.111235 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:06.116977 25705 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:06.118155 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:06.108166 25751 leader_election.cc:336] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.594s (SENT)
I20260709 12:18:06.108748 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:06.106392 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.121997 25710 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:18:06.123674 25706 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 3 to 6, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:06.126688 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:06.113860 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.114876 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.108256 27512 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:06.128818 27512 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:06.129279 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.129472 27512 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.130050 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1386) took 1281 ms. Trace:
I20260709 12:18:06.130425 25634 rpcz_store.cc:276] 0709 12:18:04.848922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.977036 (+128114us) service_pool.cc:224] Handling call
0709 12:18:06.129979 (+1152943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:06.118157 25637 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.138927 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:06.139863 26341 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.143234 26339 raft_consensus.cc:2955] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 3 to 6, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:06.156085 27479 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:06.158453 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.159381 25636 raft_consensus.cc:2376] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 2. Re-sending same reply.
I20260709 12:18:06.160075 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1697) took 1341 ms. Trace:
I20260709 12:18:06.160487 25636 rpcz_store.cc:276] 0709 12:18:04.818283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.856899 (+ 38616us) service_pool.cc:224] Handling call
0709 12:18:06.159982 (+1303083us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:06.186136 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.206231 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.206872 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.229216 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121975us
I20260709 12:18:06.232035 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:06.232278 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 113453us
W20260709 12:18:06.244313 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.257198 27517 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:06.258811 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:06.263957 27524 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:06.265357 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:06.266114 25785 raft_consensus.cc:2376] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2. Re-sending same reply.
I20260709 12:18:06.267001 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1881) took 2522 ms. Trace:
I20260709 12:18:06.267449 25704 rpcz_store.cc:276] 0709 12:18:03.744427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.629007 (+884580us) service_pool.cc:224] Handling call
0709 12:18:04.629434 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:04.629959 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.630000 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:04.667056 (+ 37056us) log.cc:844] Serialized 26 byte log entry
0709 12:18:06.266337 (+1599281us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b54000a17a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b59066b8 00007f29b598f92b 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c
0709 12:18:06.266410 (+    73us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:06.266509 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.266612 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:06.266663 (+    51us) raft_consensus.cc:1685] finished
0709 12:18:06.266679 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:06.266910 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":172,"spinlock_wait_cycles":150518016,"thread_start_us":76316,"threads_started":1,"wal-append.queue_time_us":85394,"wal-append.run_cpu_time_us":2496,"wal-append.run_wall_time_us":1059411}
I20260709 12:18:06.273798 26341 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:06.275527 25637 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:06.278939 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.280169 27524 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:06.280620 27524 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:06.281026 27524 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:06.283507 25750 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:06.311839 25712 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:06.314708 27512 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:06.316329 25710 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:06.319002 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:06.320576 25752 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:06.319836 27469 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.005s	sys 0.016s
W20260709 12:18:06.322073 27469 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.005s	sys 0.017s
I20260709 12:18:06.326891 27469 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:06.317883 27477 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:06.335582 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:06.324129 27534 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:06.339165 27534 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:06.339617 27534 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:06.340628 27505 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 1.678s	user 0.006s	sys 0.008s
I20260709 12:18:06.341786 27512 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:06.333232 27539 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:06.350167 27515 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:06.343842 25704 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:06.342886 27477 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:06.352522 27477 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:06.355434 27457 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.006s
I20260709 12:18:06.355788 27515 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:06.355952 27457 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:06.357182 26340 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:06.358712 27391 tablet_copy_client.cc:538] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:18:06.358930 27515 consensus_queue.cc:1059] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:06.362746 27479 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:06.363144 27479 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:06.364075 27479 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:06.370731 27479 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.376443 25629 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.374185 25532 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:06.381763 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:06.383698 27447 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:06.384297 27447 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:06.385197 27447 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:06.387370 27435 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:06.387778 27435 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:06.388649 27435 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:06.389817 27435 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:06.389863 25781 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
W20260709 12:18:06.401839 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 82897.6us
I20260709 12:18:06.407388 25629 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:06.358397 27457 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:06.429816 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:06.430313 26309 leader_election.cc:336] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.805s (SENT)
W20260709 12:18:06.432740 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 326929us
I20260709 12:18:06.433805 25630 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:06.437633 25630 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:06.431442 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:06.432585 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 326888us
I20260709 12:18:06.442909 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:06.444599 27539 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:06.444998 27539 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:06.445827 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.445401 27539 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.439023 25603 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:06.450951 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.440938 27433 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:18:06.411475 27546 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:18:06.458338 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.459056 25631 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.462145 27537 raft_consensus.cc:993] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:18:06.462898 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:06.463251 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.463845 25637 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.407115 27519 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:18:06.465987 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:06.470016 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.471048 25629 raft_consensus.cc:2368] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4.
I20260709 12:18:06.463714 27537 raft_consensus.cc:1081] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:18:06.475791 25634 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:18:06.474083 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 315290us
I20260709 12:18:06.481343 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:06.481207 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 51020.2us
I20260709 12:18:06.485494 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:06.485807 25633 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.488962 25632 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:06.467222 27457 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:06.429909 27534 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:18:06.502465 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45695.1us
I20260709 12:18:06.504338 25631 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:06.495024 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:06.489708 27460 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.485262 27503 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:06.491807 25630 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:06.446151 27524 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:06.495462 27540 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:06.516414 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.518527 25630 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:06.476771 27469 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:06.534435 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49251.6us
I20260709 12:18:06.535430 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:06.524875 27477 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.513816 27460 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:06.557587 25637 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
W20260709 12:18:06.559705 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 83048.9us
I20260709 12:18:06.571278 25630 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:06.575977 27503 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:06.577514 27503 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:06.602927 26309 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.895s (SENT)
I20260709 12:18:06.603549 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:06.583994 27483 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:18:06.620661 25633 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
W20260709 12:18:06.625365 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63055.2us
I20260709 12:18:06.611519 27553 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:18:06.585311 27477 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:06.592073 27562 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
W20260709 12:18:06.626627 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 116884us
I20260709 12:18:06.933579 25705 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:18:06.674511 26309 leader_election.cc:336] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.916s (SENT)
W20260709 12:18:06.912544 25659 debug-util.cc:398] Leaking SignalData structure 0x7b080032ad00 after lost signal to thread 24424
W20260709 12:18:06.701879 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 179994us
W20260709 12:18:06.694608 27544 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:18:06.915740 27435 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.005s
W20260709 12:18:06.912242 27479 consensus_meta.cc:287] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
I20260709 12:18:06.686645 25706 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:06.935995 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:06.909435 25534 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.937132 27566 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:06.706084 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:06.940389 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 249990us
I20260709 12:18:06.944609 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:18:06.937320 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08003419e0 after lost signal to thread 25570
I20260709 12:18:06.938861 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:06.732491 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.953120 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2480 after lost signal to thread 25662
W20260709 12:18:06.953616 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2460 after lost signal to thread 25663
I20260709 12:18:06.692951 27552 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:06.954627 27552 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:06.954914 25659 debug-util.cc:398] Leaking SignalData structure 0x7b0800098180 after lost signal to thread 25737
I20260709 12:18:06.955072 27552 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:06.956435 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08003997a0 after lost signal to thread 25811
W20260709 12:18:06.956796 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08003999a0 after lost signal to thread 25812
W20260709 12:18:06.939093 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.937582 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:06.940614 27435 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:06.934901 25705 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:06.958000 25659 debug-util.cc:398] Leaking SignalData structure 0x7b0800353740 after lost signal to thread 26368
I20260709 12:18:06.958837 27488 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:06.613785 27453 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:06.818048 27564 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.228s	user 0.002s	sys 0.000s
W20260709 12:18:06.942658 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.963253 25659 debug-util.cc:398] Leaking SignalData structure 0x7b08002a33c0 after lost signal to thread 27474
W20260709 12:18:06.974910 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.996109 27488 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:06.640514 27391 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:18:07.006947 27535 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:07.007409 27555 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:18:07.008230 25713 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:07.009176 27488 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:06.705194 27559 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:18:07.001746 25677 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:06.818931 27534 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:06.904767 27549 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
W20260709 12:18:06.996207 25751 leader_election.cc:336] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.735s (SENT)
W20260709 12:18:07.022347 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 347860us
I20260709 12:18:07.022771 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.023408 26309 leader_election.cc:336] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.954s (SENT)
I20260709 12:18:07.023949 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:06.915513 27515 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:06.626645 27554 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:06.730751 27524 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:06.995952 27435 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:07.027259 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:07.006462 27539 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.002s
I20260709 12:18:07.028664 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.021732 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.029964 25709 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 2: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:07.030797 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:06.958855 27567 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:07.032486 25780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:07.033846 25780 raft_consensus.cc:606] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:07.034065 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:07.032341 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.009s	sys 0.013s
W20260709 12:18:07.035404 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.009s	sys 0.013s
I20260709 12:18:07.028618 25705 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.036443 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:07.036607 25781 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:07.037086 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:07.037292 25710 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:07.002278 27562 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:18:07.038120 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:07.038573 27453 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:07.039216 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.026772 27515 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:07.040355 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.040963 27515 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.036610 25705 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:07.041574 25709 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:07.043129 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:07.044023 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.044668 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.043346 25709 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:07.038247 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:07.050560 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.047487 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45901.7us
I20260709 12:18:07.052521 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:07.054141 25706 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:07.055464 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.038s (SENT)
I20260709 12:18:07.031286 27435 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:07.063441 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.060009 25709 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.050199 27524 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:07.044004 27515 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.073333 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:07.068455 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:07.027120 27529 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:18:07.045935 27567 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:07.074572 26308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.012s	sys 0.024s
I20260709 12:18:07.032835 27539 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.083120 25677 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:07.084597 26341 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.131300 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:07.131666 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 135485us
W20260709 12:18:07.136143 25751 leader_election.cc:336] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.887s (SENT)
I20260709 12:18:07.136693 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.072397 27524 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:07.142396 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.081737 26308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.012s	sys 0.024s
W20260709 12:18:07.149822 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 629010us
I20260709 12:18:07.150141 25706 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.150813 25706 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:07.155357 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 71558.9us
I20260709 12:18:07.159921 26339 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.080098 27534 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:07.166558 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 83511.3us
I20260709 12:18:07.083751 27491 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.169250 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.083385 27567 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:07.152400 27524 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.172369 25709 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:07.171458 25706 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.174118 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.174213 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.176744 25630 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:07.176560 25638 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:07.180158 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.180474 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.180943 25638 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:07.181929 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.182672 25637 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:07.183495 27551 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.162619 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.188055 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:18:07.205317 25636 raft_consensus.cc:606] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:18:07.189055 25705 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:07.193387 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.195120 25706 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:07.198671 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 183217us
I20260709 12:18:07.195144 25709 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:07.203495 27474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4fd24cfa184aa8a8a1de8f32296e6a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:18:07.204099 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 68017.7us
I20260709 12:18:07.212630 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:07.213019 25751 leader_election.cc:336] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.910s (SENT)
I20260709 12:18:07.167907 27406 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.205824 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:07.172487 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:07.224222 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.181972 26340 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.228391 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.225167 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.242528 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.189638 27535 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:18:07.267149 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.268296 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 55298.9us
W20260709 12:18:07.269259 25751 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.922s (SENT)
I20260709 12:18:07.269718 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.271726 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 115311us
I20260709 12:18:07.276700 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:07.278318 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:07.289485 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 83712.8us
I20260709 12:18:07.236645 26338 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.241724 27440 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.236346 27447 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:07.292546 25677 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:07.308454 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.265549 27453 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:07.311589 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 87440.1us
I20260709 12:18:07.312465 26308 leader_election.cc:400] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4.
I20260709 12:18:07.312911 26308 leader_election.cc:403] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:18:07.320439 25706 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:07.322398 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:07.303217 27551 raft_consensus.cc:2764] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:18:07.293970 27523 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.011s	sys 0.024s
I20260709 12:18:07.314348 27556 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:07.332865 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 63668.4us
W20260709 12:18:07.335978 25751 leader_election.cc:336] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.931s (SENT)
I20260709 12:18:07.336349 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.337513 25751 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.941s (SENT)
I20260709 12:18:07.337746 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.323326 27524 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.340700 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.371845 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.324471 27447 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:07.330875 27453 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4.
W20260709 12:18:07.330757 27523 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.011s	sys 0.024s
I20260709 12:18:07.341437 27572 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:07.335523 27540 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:07.341487 27569 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.341184 27522 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:07.394615 27447 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:18:07.390687 27503 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:07.390868 27556 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:07.389657 27594 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:07.392252 27540 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:07.396571 27453 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:18:07.402866 27558 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.396472 27539 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:07.414687 27540 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:07.412374 27569 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:07.425982 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.428237 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 100236us
I20260709 12:18:07.420001 27524 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:07.440786 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 113929us
I20260709 12:18:07.441313 27460 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:07.450853 25706 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:07.410280 27594 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.446301 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:07.454324 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.456982 27593 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:07.458475 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:07.466339 27593 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.466961 27593 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:07.437600 27569 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.468207 27593 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.469089 27457 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:07.469300 27569 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:07.469563 27457 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:07.481850 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:07.488019 26344 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:07.482712 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:07.483042 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45354.8us
I20260709 12:18:07.489142 26339 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:07.470533 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:07.491631 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.499351 27596 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.500375 27457 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.506249 27596 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:07.511914 27483 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:07.512385 27483 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:07.513196 27483 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.514220 27483 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:07.498139 27477 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:07.509687 27447 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:18:07.522063 27552 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.566s	user 0.006s	sys 0.009s
W20260709 12:18:07.520538 27488 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.000s
I20260709 12:18:07.527840 27457 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:07.518771 27585 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:18:07.542727 27447 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:07.544394 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:07.545264 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.528331 27440 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.571035 25634 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:07.552497 26338 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
W20260709 12:18:07.576787 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.561714 27503 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:07.567584 27440 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.584338 27594 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:07.593272 27447 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.594872 26335 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:07.567409 27596 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.607013 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.610795 27572 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:07.579082 27511 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.580529 27553 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:18:07.644290 27603 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.143s	user 0.000s	sys 0.002s
I20260709 12:18:07.644439 27553 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:07.622889 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.623654 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.554713 27488 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.624408 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.655963 25635 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:07.625844 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:07.626544 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.628193 27535 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.630077 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.631222 25712 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.631803 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.660605 25634 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:07.632843 25532 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 191) took 1309 ms. Trace:
I20260709 12:18:07.554370 27552 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.628242 25533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 301) took 3235 ms. Trace:
W20260709 12:18:07.614773 27435 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.000s
I20260709 12:18:07.623588 27540 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.650797 27579 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.669564 27579 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.652887 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:07.670830 25633 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:18:07.670997 25631 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:07.644291 27511 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.654291 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:07.679564 25637 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.665408 25638 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.667146 25629 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
W20260709 12:18:07.668788 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.672606 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.661376 25532 rpcz_store.cc:276] 0709 12:18:06.323426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.323579 (+   153us) service_pool.cc:224] Handling call
0709 12:18:06.371107 (+ 47528us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:07.632748 (+1261641us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:06.509586 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:06.618645 (+109059us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:06.622597 (+  3952us) write_op.cc:435] Acquired schema lock
0709 12:18:06.626849 (+  4252us) tablet.cc:662] Decoding operations
0709 12:18:06.722173 (+ 95324us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:06.730937 (+  8764us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:06.734411 (+  3474us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:18:06.853447 (+119036us) tablet.cc:701] Row locks acquired
0709 12:18:06.941517 (+ 88070us) write_op.cc:260] PREPARE: finished
0709 12:18:06.941726 (+   209us) write_op.cc:270] Start()
0709 12:18:06.941893 (+   167us) write_op.cc:276] Timestamp: P: 1783599486941671 usec, L: 0
0709 12:18:06.941928 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:18:06.945612 (+  3684us) log.cc:844] Serialized 536 byte log entry
0709 12:18:06.948184 (+  2572us) op_driver.cc:464] REPLICATION: finished
0709 12:18:06.948622 (+   438us) write_op.cc:301] APPLY: starting
0709 12:18:06.948704 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:06.948810 (+   106us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:06.948961 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:06.949188 (+   227us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:06.949319 (+   131us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:18:06.949361 (+    42us) write_op.cc:312] APPLY: finished
0709 12:18:06.949504 (+   143us) log.cc:844] Serialized 23 byte log entry
0709 12:18:06.949648 (+   144us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:06.949711 (+    63us) write_op.cc:454] Released schema lock
0709 12:18:06.949747 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:07.439679 (+489932us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598ae28 00007f29b93c5581 00007f29b93c4d58 00007f29b93c99f1 00007f29b93c99a9 00007f29b93c9939 00007f29b93c9901 00007f29b93c8b2d 0000000000388634 0000000000388479
0709 12:18:07.439909 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:07.439946 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":680983,"reported_tablets":1,"spinlock_wait_cycles":14981504,"child_traces":[["op",{"apply.queue_time_us":153,"apply.run_cpu_time_us":1419,"apply.run_wall_time_us":1420,"num_ops":1,"prepare.queue_time_us":75582,"prepare.run_cpu_time_us":22827,"prepare.run_wall_time_us":943680,"replication_time_us":6050,"spinlock_wait_cycles":6621568,"thread_start_us":41443,"threads_started":1}]]}
I20260709 12:18:07.688320 27440 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:07.689078 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:07.655792 27522 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:07.655285 27553 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.693574 27488 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:07.695328 25630 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.697348 27583 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.697847 27552 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:07.700344 27553 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:07.703063 25637 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.703619 25637 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:07.706228 27579 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:07.706291 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.706950 27579 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:07.710603 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:07.716645 27540 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:07.667670 27535 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.617235 27567 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:07.720712 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 94316.1us
I20260709 12:18:07.719884 27535 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:07.675160 27435 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.723373 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:07.727360 25629 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.664285 25533 rpcz_store.cc:276] 0709 12:18:04.392579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.392768 (+   189us) service_pool.cc:224] Handling call
0709 12:18:07.628125 (+3235357us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:04.484801 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:04.485470 (+   669us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:04.485514 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:18:04.485542 (+    28us) tablet.cc:662] Decoding operations
0709 12:18:04.485899 (+   357us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:04.485990 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:04.486012 (+    22us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:18:04.486388 (+   376us) tablet.cc:701] Row locks acquired
0709 12:18:04.486455 (+    67us) write_op.cc:260] PREPARE: finished
0709 12:18:04.501639 (+ 15184us) write_op.cc:270] Start()
0709 12:18:04.527023 (+ 25384us) write_op.cc:276] Timestamp: P: 1783599484501054 usec, L: 0
0709 12:18:04.527070 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 12:18:04.527825 (+   755us) log.cc:844] Serialized 541 byte log entry
0709 12:18:06.339650 (+1811825us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:06.605908 (+266258us) op_driver.cc:464] REPLICATION: finished
0709 12:18:06.945798 (+339890us) write_op.cc:301] APPLY: starting
0709 12:18:06.945903 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:06.946014 (+   111us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:06.946052 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:06.946415 (+   363us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:06.946725 (+   310us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:18:06.946769 (+    44us) write_op.cc:312] APPLY: finished
0709 12:18:06.947012 (+   243us) log.cc:844] Serialized 23 byte log entry
0709 12:18:06.947200 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:06.947276 (+    76us) write_op.cc:454] Released schema lock
0709 12:18:06.947331 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:06.947769 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:06.947799 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":402007,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":332947,"apply.run_cpu_time_us":1923,"apply.run_wall_time_us":1923,"num_ops":1,"prepare.queue_time_us":20584,"prepare.run_cpu_time_us":65849,"prepare.run_wall_time_us":187577,"raft.queue_time_us":185478,"raft.run_cpu_time_us":7672,"raft.run_wall_time_us":409105,"replication_time_us":2030877,"spinlock_wait_cycles":106268288,"thread_start_us":531657,"threads_started":5,"wal-append.queue_time_us":134586}]]}
I20260709 12:18:07.745645 25636 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:07.746991 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.748725 25710 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:07.762802 25751 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.865s (SENT)
I20260709 12:18:07.770771 25631 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
W20260709 12:18:07.748960 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 425967us
W20260709 12:18:07.782126 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 515058us
I20260709 12:18:07.783061 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.797091 25637 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:07.799786 25751 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.886s (SENT)
I20260709 12:18:07.800470 27567 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:07.807840 25629 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:07.811422 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.828480 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45494.3us
I20260709 12:18:07.850914 27565 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:07.850992 25637 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.851716 27565 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:07.852238 27565 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.852968 27570 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:07.836272 27477 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:07.857059 25712 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:18:07.857889 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46584.8us
I20260709 12:18:07.854390 27546 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:18:07.859733 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 60027us
I20260709 12:18:07.827020 27535 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:07.864504 25637 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:18:07.890779 25751 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.006s (SENT)
I20260709 12:18:07.895447 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:07.896485 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.896590 25637 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:07.835974 27479 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:18:07.903427 25677 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:07.864614 27546 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:07.909619 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.909480 27546 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:07.863268 27511 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:07.848824 27552 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
W20260709 12:18:07.916433 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51140.7us
I20260709 12:18:07.916779 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.917309 25751 leader_election.cc:336] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.019s (SENT)
I20260709 12:18:07.917898 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:07.910801 27546 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.929414 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:07.929667 27534 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:07.930279 25710 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:07.934624 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 12:18:07.939044 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 96348.3us
I20260709 12:18:07.940291 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:07.945950 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:07.947863 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:07.953754 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.959117 25751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.027s (SENT) [suppressed 79 similar messages]
W20260709 12:18:07.965507 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 62217.1us
I20260709 12:18:07.966338 25677 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:07.967262 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.959069 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:07.811169 27519 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:18:07.947088 27523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.009s
I20260709 12:18:07.973184 25677 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:07.948946 27511 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.980897 25637 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
W20260709 12:18:07.986516 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.964350 27435 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:07.931993 27534 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:07.965737 27503 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:07.985055 27523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.010s	sys 0.010s
I20260709 12:18:07.995797 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:07.999403 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.994272 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 348412us
I20260709 12:18:08.005785 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:07.974865 27594 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:07.982414 27585 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.988023 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:07.984073 27562 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.992974 27540 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:07.969182 27483 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:18:08.014889 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.993160 27566 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:08.026209 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.020382 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:07.993376 27535 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:08.046303 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.009680 27594 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:08.055254 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.045392 27585 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.045715 27524 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:08.025413 27520 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:08.068862 25677 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:08.044273 27483 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:08.010344 27503 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.055745 27522 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:08.058175 27539 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:08.067628 27594 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:08.097456 25785 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:08.097534 25677 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:08.079886 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 431257us
W20260709 12:18:08.149633 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52169.9us
I20260709 12:18:08.151664 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.150095 25750 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:08.085270 27566 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:08.067481 27506 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:18:08.155296 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:08.080466 27552 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:18:08.162717 27568 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:08.097255 27567 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:08.114904 26337 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:08.173540 25780 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:08.174131 27520 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:08.175312 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 558) took 1108 ms. Trace:
I20260709 12:18:08.175720 26337 rpcz_store.cc:276] 0709 12:18:07.066943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.075319 (+  8376us) service_pool.cc:224] Handling call
0709 12:18:07.633753 (+558434us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:07.633963 (+   210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:08.173669 (+539706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.175049 (+  1380us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54005a7154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:08.175228 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062327296}
I20260709 12:18:08.097995 27524 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.180249 27524 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.157302 27570 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:18:08.183043 27503 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.172168 27453 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:08.172425 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.172533 27469 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:08.173225 26338 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:18:08.188167 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 229095us
I20260709 12:18:08.193342 26341 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:08.190537 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:08.163297 27602 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:08.167855 27568 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.215919 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.215211 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.222173 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.184213 27567 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:08.187875 27604 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:08.221905 27503 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.166719 27540 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:18:08.196560 27557 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:08.256548 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 316376us
I20260709 12:18:08.257081 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.213219 27602 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.262630 25786 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:08.223042 25780 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
I20260709 12:18:08.191658 27453 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.269564 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:08.283272 25637 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:08.169531 27483 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.268745 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.287029 25782 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 2: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:08.275668 25629 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:08.276772 27614 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:18:08.278862 27606 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:18:08.280738 26336 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:08.240867 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:08.281612 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:08.167420 27566 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.285331 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:08.223132 27568 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:08.280889 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:08.266696 27602 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:08.268110 27567 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:08.299187 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.240758 27522 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:08.257339 25778 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:18:08.325955 25706 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:08.256443 27503 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.323325 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.244601 27523 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
W20260709 12:18:08.308058 27474 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.003s
I20260709 12:18:08.353569 27474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56146 (request call id 371) took 6008 ms. Trace:
I20260709 12:18:08.282235 27579 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 12:18:08.355271 27616 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:18:08.326960 26341 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:08.337121 27522 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.342566 27554 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.366465 27554 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.368055 27522 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:08.307047 26335 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
W20260709 12:18:08.344956 27612 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:18:08.345772 27523 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:08.304675 27534 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:08.351661 27483 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:08.281380 27557 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.370471 25780 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:08.379343 26344 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:08.332027 26340 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:08.383845 27585 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.318090 27611 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:18:08.385591 27611 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:08.386026 27611 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:08.387557 25786 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:08.368891 27618 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:18:08.393142 27609 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.206s	user 0.003s	sys 0.000s
I20260709 12:18:08.395290 25787 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
W20260709 12:18:08.395148 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.305109 27540 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:18:08.400997 25784 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:08.363997 27469 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.373057 27604 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:08.416623 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.418064 25781 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:08.421932 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:08.422701 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.423393 25780 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:08.400882 27506 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:08.425580 27506 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.421967 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:08.428408 27506 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.429164 27503 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.429589 27572 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:08.430105 27572 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:08.430294 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.430820 27453 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.431058 27540 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:08.431241 27572 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.415107 27469 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.435201 25780 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:08.435542 27469 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:08.437641 27557 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:08.401180 27553 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.692s	user 0.001s	sys 0.011s
I20260709 12:18:08.442428 27524 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:08.442870 27524 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:08.443650 27524 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.445876 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.377667 27523 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:08.447292 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.447335 27523 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.448031 25781 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:08.448855 27523 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:08.449198 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:08.449872 27605 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.288s	user 0.000s	sys 0.005s
I20260709 12:18:08.450930 27611 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.452383 27611 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:08.454103 27604 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.455376 26340 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:08.456441 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:08.422745 27554 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:08.460439 27391 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:18:08.461102 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:08.461566 27391 tablet_bootstrap.cc:492] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
W20260709 12:18:08.477885 25751 connection.cc:256] RPC call timeout handler was delayed by -1.0566268640000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
W20260709 12:18:08.482705 25751 connection.cc:256] RPC call timeout handler was delayed by -1.025126863999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 12:18:08.483127 25751 connection.cc:256] RPC call timeout handler was delayed by -1.0134268640001665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
W20260709 12:18:08.483716 25751 leader_election.cc:336] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.032s (SENT)
I20260709 12:18:08.485328 25786 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 12:18:08.485896 25786 pending_rounds.cc:85] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 3
I20260709 12:18:08.489114 27539 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:08.489574 27539 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:08.490819 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:08.486249 25786 pending_rounds.cc:107] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
W20260709 12:18:08.496985 25751 leader_election.cc:336] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.048s (SENT)
I20260709 12:18:08.497581 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:08.498661 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 327798us
I20260709 12:18:08.501605 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:08.435083 27566 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:08.368696 27522 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 12:18:08.505462 25786 raft_consensus.cc:2917] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 12:18:08.484246 27391 ts_tablet_manager.cc:1403] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 1.867s	user 0.037s	sys 0.028s
W20260709 12:18:08.508538 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.509964 25786 raft_consensus.cc:2889] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:08.511158 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:08.512820 25751 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.061s (SENT)
I20260709 12:18:08.496142 27572 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:08.506335 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 348407us
I20260709 12:18:08.509307 27557 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:18:08.537561 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.539726 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 249435us
I20260709 12:18:08.541020 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:08.542932 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 330705us
I20260709 12:18:08.354106 27474 rpcz_store.cc:276] 0709 12:18:02.345072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.345887 (+   815us) service_pool.cc:224] Handling call
0709 12:18:04.457995 (+2112108us) ts_tablet_manager.cc:934] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
0709 12:18:07.202113 (+2744118us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:07.716291 (+514178us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:07.716403 (+   112us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:08.352420 (+636017us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:08.352867 (+   447us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:08.353299 (+   432us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:08.353442 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":336,"tablet-copy.queue_time_us":1113751,"thread_start_us":1075891,"threads_started":2}
W20260709 12:18:08.567972 25751 leader_election.cc:336] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.174s (SENT)
I20260709 12:18:08.572717 27538 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:08.576507 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:08.584050 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:08.587123 25787 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:08.589375 27557 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:08.512586 27539 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.591032 27539 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:08.536237 27587 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 12:18:08.584376 27546 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.002s
I20260709 12:18:08.542954 27598 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:18:08.606604 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:08.585690 27474 tablet_copy_client.cc:806] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:18:08.586820 27522 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:08.567514 27594 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.621677 25709 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:08.622931 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.637652 25711 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:08.639883 26339 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:18:08.643944 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 108697us
I20260709 12:18:08.645535 27624 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:08.646545 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.647253 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.646445 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 105433us
W20260709 12:18:08.648175 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.648737 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.649070 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.651575 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.652868 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.653213 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.653801 25784 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:08.661602 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 166963us
I20260709 12:18:08.622514 27522 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:08.608666 27447 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:18:08.671793 27474 tablet_copy_client.cc:670] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:18:08.646212 27624 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:08.690214 27614 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:18:08.687398 27572 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:18:08.693521 27565 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.841s	user 0.012s	sys 0.004s
I20260709 12:18:08.630309 27566 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:18:08.688850 27562 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:08.729596 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76235.3us
I20260709 12:18:08.732409 27391 raft_consensus.cc:359] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:08.733321 27391 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
W20260709 12:18:08.726672 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 180683us
I20260709 12:18:08.681744 27604 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:08.752942 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.753978 27628 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:18:08.735489 25751 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.178s (SENT)
I20260709 12:18:08.760102 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:08.760603 25782 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
W20260709 12:18:08.762550 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 84141.2us
W20260709 12:18:08.768968 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.788921 27644 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:18:08.789746 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54207.5us
W20260709 12:18:08.794227 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.832659 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.833230 25601 connection.cc:256] RPC call timeout handler was delayed by -2.8352156509999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 12:18:08.833624 25601 connection.cc:256] RPC call timeout handler was delayed by -2.805515651000178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 12:18:08.833966 25601 connection.cc:256] RPC call timeout handler was delayed by -2.601215651000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
W20260709 12:18:08.834348 25601 connection.cc:256] RPC call timeout handler was delayed by -2.571515651000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
W20260709 12:18:08.834738 25601 connection.cc:256] RPC call timeout handler was delayed by -2.5625156510000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 12:18:08.832285 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 158851us
W20260709 12:18:08.835141 25601 connection.cc:256] RPC call timeout handler was delayed by -2.5166156510001656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
W20260709 12:18:08.835893 25601 connection.cc:256] RPC call timeout handler was delayed by -2.5022156510001423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 12:18:08.836710 25601 connection.cc:256] RPC call timeout handler was delayed by -2.3879156510001849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
W20260709 12:18:08.837404 25601 connection.cc:256] RPC call timeout handler was delayed by -2.3870156510001834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 12:18:08.837975 25601 connection.cc:256] RPC call timeout handler was delayed by -2.3852156510001805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
W20260709 12:18:08.838338 25601 connection.cc:256] RPC call timeout handler was delayed by -2.3717156510001587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
W20260709 12:18:08.838960 25601 connection.cc:256] RPC call timeout handler was delayed by -2.3636156510001456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
W20260709 12:18:08.839924 25601 connection.cc:256] RPC call timeout handler was delayed by -2.3519156510001267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 12:18:08.844476 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.851619 25677 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:08.849793 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 161295us
I20260709 12:18:08.741142 27562 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.728487 27565 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.837411 27644 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.860466 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:08.861479 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.864847 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:08.861552 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.863483 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:08.739710 27522 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.855063 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:08.878584 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:08.882397 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:08.883291 25787 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:08.768673 27391 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:08.853483 27566 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:18:08.892135 27552 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:08.896831 27623 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:08.897053 27522 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:08.881583 27483 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.903035 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:08.854385 27520 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:18:08.906764 26340 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 12:18:08.917061 26340 pending_rounds.cc:85] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 3
I20260709 12:18:08.936453 27647 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 12:18:08.947185 27631 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:18:08.947305 27440 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:08.947923 27440 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:08.889585 27644 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.898201 27594 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.945550 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:08.954854 27440 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.957919 27440 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:08.958271 27644 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:08.943776 26340 pending_rounds.cc:107] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
W20260709 12:18:08.968964 27632 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:18:08.929224 27623 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:08.949150 27519 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.870s	user 0.005s	sys 0.015s
I20260709 12:18:08.915594 27483 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:08.944774 27520 consensus_queue.cc:1243] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:08.975504 27641 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 12:18:08.891865 27639 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:18:08.976284 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:08.978899 27624 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:08.986817 27579 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:08.987443 27579 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:08.988545 27524 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:08.994884 27630 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
I20260709 12:18:08.967841 27552 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:08.995047 27569 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:18:09.001592 27620 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:08.970741 26340 raft_consensus.cc:2917] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 12:18:09.010828 27594 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:09.013818 26340 raft_consensus.cc:2889] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:09.018922 27565 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:08.981827 27520 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:18:09.014139 27633 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:18:09.027302 27639 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:18:09.044567 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 71663.5us
I20260709 12:18:09.045779 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:09.132733 27629 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:09.133342 27629 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:09.030612 27647 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:09.134500 27629 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:09.155339 27602 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.794s	user 0.006s	sys 0.010s
I20260709 12:18:09.158581 25638 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:09.037523 27457 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:09.178320 27457 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.181277 27457 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:09.182811 25781 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:09.184952 27568 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.865s	user 0.012s	sys 0.001s
I20260709 12:18:09.185505 27568 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:09.189468 27539 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.000s
W20260709 12:18:09.193043 27567 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.861s	user 0.013s	sys 0.002s
I20260709 12:18:09.193574 27567 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.110157 27506 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.053920 27639 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:09.221241 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.162812 27483 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:09.203284 25638 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:09.124951 27620 raft_consensus.cc:2764] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 12:18:09.204048 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 158416us
I20260709 12:18:09.065374 27453 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:09.159302 27647 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:09.156473 27562 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:09.199436 25707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.040s	sys 0.091s
I20260709 12:18:09.234071 27639 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.251421 25779 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 5. Ops: []
I20260709 12:18:09.265061 25601 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:09.206589 27602 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:09.268975 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 468192us
I20260709 12:18:09.275346 25706 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:09.279363 27567 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:09.253114 27453 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.253531 27629 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:09.227453 27440 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:09.285293 27440 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.288173 27440 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:09.252676 25707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.040s	sys 0.092s
I20260709 12:18:09.295019 27538 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:09.295717 27538 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:09.225558 27506 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.271714 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:09.319633 25709 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:09.326714 25780 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 3. Ops: []
I20260709 12:18:09.325438 27391 ts_tablet_manager.cc:1434] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.814s	user 0.005s	sys 0.006s
W20260709 12:18:09.328322 27391 ts_tablet_manager.cc:1467] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Tablet startup took 817ms
I20260709 12:18:09.299817 27620 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:09.321312 27639 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:09.323287 27611 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.870s	user 0.016s	sys 0.001s
I20260709 12:18:09.264505 27623 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.309923 26335 raft_consensus.cc:1217] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:18:09.328390 25778 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:18:09.341084 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 80128.1us
W20260709 12:18:09.349085 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 482291us
I20260709 12:18:09.335714 27620 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:09.360694 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.340525 27469 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:09.348416 25708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.007s	sys 0.037s
I20260709 12:18:09.375641 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:09.346218 27638 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:09.348619 27618 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.378242 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.350026 25778 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:09.392378 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:09.396029 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.396631 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.328223 27647 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.375869 27620 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:09.409776 25706 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:09.394979 25708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.011s	sys 0.037s
I20260709 12:18:09.413064 27440 raft_consensus.cc:2764] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:18:09.419727 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1856) took 1331 ms. Trace:
I20260709 12:18:09.394457 27618 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:09.425971 25783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.024s	sys 0.031s
I20260709 12:18:09.420186 25707 rpcz_store.cc:276] 0709 12:18:08.088643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.088963 (+   320us) service_pool.cc:224] Handling call
0709 12:18:08.111148 (+ 22185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:08.536673 (+425525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:08.536710 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:08.559793 (+ 23083us) log.cc:844] Serialized 26 byte log entry
0709 12:18:09.289323 (+729530us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:09.290974 (+  1651us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.291232 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.291330 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:09.419370 (+128040us) raft_consensus.cc:1685] finished
0709 12:18:09.419391 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.419657 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1908,"prepare.queue_time_us":975,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":3438,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":40772,"thread_start_us":1073728,"threads_started":2,"wal-append.queue_time_us":808823}
W20260709 12:18:09.432955 26339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.011s	sys 0.019s
W20260709 12:18:09.434157 26339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.012s	sys 0.019s
I20260709 12:18:09.401079 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:09.432885 25783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.024s	sys 0.032s
W20260709 12:18:09.436378 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:09.329823 27391 ts_tablet_manager.cc:1469] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Trace:
0709 12:17:57.900484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.900638 (+   154us) service_pool.cc:224] Handling call
0709 12:17:57.901792 (+  1154us) ts_tablet_manager.cc:934] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
0709 12:18:01.962744 (+4060952us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:02.249820 (+287076us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:02.249973 (+   153us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:03.377200 (+1127227us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:03.377553 (+   353us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:03.813894 (+436341us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:03.813960 (+    66us) inbound_call.cc:177] Queueing success response
0709 12:18:06.616977 (+2803017us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:18:08.515139 (+1898162us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:18:08.729717 (+214578us) tablet_replica.cc:261] Starting instrumentation
0709 12:18:08.730417 (+   700us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":263,"mutex_wait_us":2295,"raft.queue_time_us":3405,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":14168,"tablet-copy.queue_time_us":257,"tablet-download-pool.queue_time_us":12317,"tablet-download-pool.run_cpu_time_us":4651,"tablet-download-pool.run_wall_time_us":1710936,"thread_start_us":2205971,"threads_started":1}
W20260709 12:18:09.444151 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.011s	sys 0.014s
I20260709 12:18:09.444378 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1839) took 1110 ms. Trace:
W20260709 12:18:09.449285 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:09.444538 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.011s	sys 0.014s
W20260709 12:18:09.450352 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 581117us
W20260709 12:18:09.450906 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 88438.9us
I20260709 12:18:09.416544 27568 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:09.406106 27538 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.425661 27602 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:09.462142 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.444768 25785 rpcz_store.cc:276] 0709 12:18:08.333738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.350723 (+ 16985us) service_pool.cc:224] Handling call
0709 12:18:08.351170 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:08.439691 (+ 88521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:08.439734 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:08.462915 (+ 23181us) log.cc:844] Serialized 26 byte log entry
0709 12:18:08.691179 (+228264us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:08.734760 (+ 43581us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.756554 (+ 21794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.329881 (+573327us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b1800337600. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:09.443804 (+113923us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b54002b4654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:09.443865 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:09.443908 (+    43us) raft_consensus.cc:1685] finished
0709 12:18:09.443922 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.444311 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20204,"prepare.queue_time_us":158001,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":47598,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":14516,"spinlock_wait_cycles":197591168,"thread_start_us":234044,"threads_started":2,"wal-append.queue_time_us":323079}
I20260709 12:18:09.462953 25629 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:09.459106 25638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:18:09.464020 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.464917 25638 raft_consensus.cc:606] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership
I20260709 12:18:09.475286 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.476215 25636 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:09.416129 27469 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.423370 27656 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:09.455580 27647 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:09.485673 25711 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:09.487498 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.454722 27562 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:09.492003 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.496032 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:09.496816 25712 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:09.500278 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:09.500766 27562 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.499967 27568 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:09.466120 27587 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:18:09.514158 25601 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.488826 27552 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:09.512279 26343 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:09.490339 27506 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:09.523257 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.498080 27602 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:09.594240 27620 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.595841 27483 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:18:09.599195 25784 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.001s	sys 0.018s
I20260709 12:18:09.588726 27562 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:09.512131 27570 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:09.585853 27565 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:18:09.599931 25784 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.002s	sys 0.019s
W20260709 12:18:09.601042 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 86987.3us
W20260709 12:18:09.583076 27524 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.594s	user 0.013s	sys 0.000s
W20260709 12:18:09.607887 27511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.005s	sys 0.014s
I20260709 12:18:09.610170 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:09.599422 27594 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:09.605326 27572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.007s	sys 0.019s
I20260709 12:18:09.594321 27602 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:09.606803 27488 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:09.622949 25705 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:09.498572 27656 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:09.622912 27511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.005s	sys 0.014s
W20260709 12:18:09.640084 25710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.016s	sys 0.002s
W20260709 12:18:09.626044 27572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.007s	sys 0.019s
W20260709 12:18:09.646183 27546 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.005s	sys 0.016s
I20260709 12:18:09.650553 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.638190 27506 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:09.630537 27503 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.052s	sys 0.105s
I20260709 12:18:09.638624 27440 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:09.648428 25710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.017s	sys 0.003s
I20260709 12:18:09.631660 27488 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:09.664480 25677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.000s	sys 0.021s
W20260709 12:18:09.665534 27519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.038s	sys 0.060s
I20260709 12:18:09.654417 27572 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:09.671207 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.001s	sys 0.024s
W20260709 12:18:09.694049 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.002s	sys 0.025s
W20260709 12:18:09.671149 25677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.000s	sys 0.022s
W20260709 12:18:09.663081 27546 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.006s	sys 0.016s
W20260709 12:18:09.694408 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 814263us
W20260709 12:18:09.696616 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46203.1us
W20260709 12:18:09.697165 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.697017 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:09.695256 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 843674us
I20260709 12:18:09.667210 26338 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
W20260709 12:18:09.668427 27503 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.053s	sys 0.106s
I20260709 12:18:09.668304 27620 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:09.676702 27566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.009s	sys 0.021s
W20260709 12:18:09.676538 27519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.038s	sys 0.061s
I20260709 12:18:09.676535 27565 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:09.683652 27453 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:09.706670 27524 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:09.710893 27524 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:09.711385 27524 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:09.719161 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.721724 27538 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:09.729110 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:09.733098 25712 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:09.670809 27645 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 12:18:09.734615 26341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.010s
W20260709 12:18:09.735198 26341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.012s	sys 0.011s
W20260709 12:18:09.723631 27566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.010s	sys 0.022s
I20260709 12:18:09.704454 27469 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:09.670277 27506 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.744076 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:09.758211 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 51252.4us
I20260709 12:18:09.765347 27506 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:09.766225 25704 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:09.762122 27638 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:09.770428 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 396) took 1354 ms. Trace:
I20260709 12:18:09.770939 26340 rpcz_store.cc:276] 0709 12:18:08.416310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.418467 (+  2157us) service_pool.cc:224] Handling call
0709 12:18:08.422275 (+  3808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.019647 (+597372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.770131 (+750484us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b540056ead4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:09.770358 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75709312}
I20260709 12:18:09.773429 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1391) took 1330 ms. Trace:
I20260709 12:18:09.777232 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.777921 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.778393 25638 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
W20260709 12:18:09.779899 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.782452 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:09.783334 25638 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:09.773821 25780 rpcz_store.cc:276] 0709 12:18:08.442850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.449904 (+  7054us) service_pool.cc:224] Handling call
0709 12:18:08.450409 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.327340 (+876931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.766373 (+439033us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54000866d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:09.773360 (+  6987us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014630144}
I20260709 12:18:09.785298 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:09.786374 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 256) took 1136 ms. Trace:
I20260709 12:18:09.786630 25601 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.786856 25781 rpcz_store.cc:276] 0709 12:18:08.650066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.650858 (+   792us) service_pool.cc:224] Handling call
0709 12:18:08.651193 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.183563 (+532370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.786137 (+602574us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54001257d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:09.786320 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221948288}
I20260709 12:18:09.793071 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.801748 25630 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
W20260709 12:18:09.795392 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 272173us
I20260709 12:18:09.796797 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:09.758277 27644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.015s	sys 0.015s
I20260709 12:18:09.813122 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.814204 25705 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:09.784624 27568 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:09.784822 27506 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:09.819245 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.817508 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.819588 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:09.820041 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 152510us
W20260709 12:18:09.810051 27644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.016s	sys 0.015s
I20260709 12:18:09.789644 27664 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:09.827581 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.831810 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.832983 27474 tablet_copy_client.cc:538] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:18:09.833840 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.835034 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.868860 25704 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:09.869374 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:09.870092 25707 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:09.835906 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:09.836427 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:09.830593 27506 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.840996 27656 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:09.848055 27655 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 12:18:09.849309 27667 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:18:09.821987 27568 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.811690 27566 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.858108 27569 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:09.859040 27581 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:09.807534 27538 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.794271 27661 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:09.811205 27639 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:09.807395 27668 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:09.762265 27572 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:09.865964 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.866750 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:09.852137 27652 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:18:09.867870 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:09.873641 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50922.1us
I20260709 12:18:09.865408 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:09.881546 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 62371us
I20260709 12:18:09.917639 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:09.917881 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:09.905982 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 310806us
I20260709 12:18:09.867213 27554 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.864822 27620 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:09.864745 27477 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:09.922533 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54726.3us
W20260709 12:18:09.924110 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58226.7us
W20260709 12:18:09.927937 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 95690.9us
I20260709 12:18:09.886543 27664 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:09.929126 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:09.908080 27581 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:09.930199 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.937284 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:09.937827 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:09.990521 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.939884 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:09.927377 27477 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:09.946874 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:09.926898 27554 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.911928 27566 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.994220 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:09.948215 27672 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:09.930567 27565 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:09.988698 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 70888.1us
I20260709 12:18:09.949155 25632 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:09.999143 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:09.999470 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.998152 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:09.946733 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:09.989463 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.914248 27569 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.929396 27680 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:10.001861 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:09.992985 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62895.9us
I20260709 12:18:09.995277 25633 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:09.995440 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.935230 27457 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:09.987107 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:09.917459 27538 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.007143 25635 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:09.921185 27639 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:09.927428 27668 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:09.988443 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:09.929425 27666 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:10.003031 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:09.934617 27620 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:09.992743 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1861) took 1389 ms. Trace:
I20260709 12:18:10.017431 25632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:18:10.047420 25632 raft_consensus.cc:606] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership
I20260709 12:18:10.031430 27653 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:10.032052 27673 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:18:10.033645 25629 raft_consensus.cc:2410] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:18:10.009805 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:09.978720 27644 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:10.036041 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:09.992498 27567 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:10.043798 25635 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:09.992110 27440 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:10.018590 27546 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:09.996677 27506 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:09.978377 27581 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:10.049782 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60427.2us
I20260709 12:18:10.053079 27625 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:10.031966 27638 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:09.996860 27652 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 3 LEARNER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:09.987255 27594 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:18:10.011080 27457 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.008037 27611 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:10.081858 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:10.083483 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 263) took 1217 ms. Trace:
I20260709 12:18:10.060034 25781 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:10.053879 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.063832 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:10.064077 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:10.066080 25751 leader_election.cc:336] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.668s (SENT)
W20260709 12:18:10.088479 25751 leader_election.cc:336] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.720s (SENT)
I20260709 12:18:10.089022 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.069123 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.043346 27680 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.034912 27666 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:10.059156 27469 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:18:10.072580 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.073019 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:10.075908 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.077425 26344 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:10.084004 25784 rpcz_store.cc:276] 0709 12:18:08.866355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.866534 (+   179us) service_pool.cc:224] Handling call
0709 12:18:08.866920 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:09.614628 (+747708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.614661 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:09.643990 (+ 29329us) log.cc:844] Serialized 26 byte log entry
0709 12:18:09.817403 (+173413us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:09.817437 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.817504 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.083238 (+265734us) raft_consensus.cc:1685] finished
0709 12:18:10.083256 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.083410 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43468,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":9666,"thread_start_us":889652,"threads_started":2,"wal-append.queue_time_us":283031}
I20260709 12:18:10.059122 27665 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:10.103438 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:10.106693 25787 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:10.108127 25751 connection.cc:256] RPC call timeout handler was delayed by -1.0253518369997892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 12:18:10.108898 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:10.109460 25781 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:10.109858 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.110167 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:10.112016 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:10.083402 27457 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:10.062608 27519 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:18:10.058945 27546 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.057210 27653 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.125381 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.082116 27611 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:10.056315 27440 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:10.059765 27567 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.102272 27666 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:10.115979 27644 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:10.132474 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.126219 27651 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:10.137821 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76369.8us
I20260709 12:18:10.138077 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:10.136539 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:10.133517 25778 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.722s	user 0.014s	sys 0.000s
I20260709 12:18:10.146394 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:10.076056 27638 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:10.135334 26340 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:10.116818 27678 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.135432 27611 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:10.131229 27624 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:10.169857 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.172078 27678 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.172627 27653 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:10.173249 27653 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:10.173897 27653 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:10.174477 27440 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.100754 27477 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.170572 25783 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:10.128049 27524 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.179131 25636 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:10.102952 27664 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.179023 27569 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.126395 27661 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:18:10.119871 27665 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.154744 27568 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.188719 25781 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:10.190137 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.159795 27676 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.112s
I20260709 12:18:10.047211 25708 rpcz_store.cc:276] 0709 12:18:08.600052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.602053 (+  2001us) service_pool.cc:224] Handling call
0709 12:18:08.693351 (+ 91298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:09.436961 (+743610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.473259 (+ 36298us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:09.590201 (+116942us) log.cc:844] Serialized 26 byte log entry
0709 12:18:09.809857 (+219656us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:09.816863 (+  7006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.822970 (+  6107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:09.935163 (+112193us) raft_consensus.cc:1685] finished
0709 12:18:09.935192 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.989426 (+ 54234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31632,"raft.queue_time_us":142657,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":10108,"thread_start_us":852941,"threads_started":2,"wal-append.queue_time_us":324945}
I20260709 12:18:10.137929 27620 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.194877 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:10.211392 25706 raft_consensus.cc:1813] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Handling vote request from an unknown peer 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:10.211850 25706 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:10.212886 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:10.198153 25787 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:10.202665 25638 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:10.208144 25780 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:10.156068 27594 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:18:10.208989 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:10.215824 27684 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:18:10.196398 27520 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:10.221758 27656 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:10.227587 27546 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.132406 27668 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.176175 27624 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.230787 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:10.233394 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.241293 25709 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:10.155555 25778 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.255048 27694 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:10.257872 25785 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:10.230554 27477 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.209774 27569 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:18:10.228658 27520 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:10.225742 27538 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:10.310730 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 274753us
I20260709 12:18:10.320556 25706 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:10.325525 25706 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:18:10.330165 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 251144us
I20260709 12:18:10.306401 27554 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.310833 27566 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.343441 27690 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.313961 27572 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:10.149549 27651 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.318743 27602 raft_consensus.cc:2764] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:18:10.321094 27639 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.365059 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.376919 25636 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:10.377950 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.344347 27553 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:10.379041 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.385543 25636 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:10.388070 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.351770 27594 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.388985 25636 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
W20260709 12:18:10.375034 27647 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.888s	user 0.014s	sys 0.000s
I20260709 12:18:10.320091 27694 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.362546 27567 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:10.380357 27618 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.955s	user 0.013s	sys 0.003s
I20260709 12:18:10.373502 27523 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.431509 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.458178 25633 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 2: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:10.446153 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:10.444829 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 244579us
I20260709 12:18:10.363190 27572 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.458060 27579 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.501101 25706 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:10.440793 27651 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:10.511075 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 258863us
I20260709 12:18:10.458012 27594 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.443491 27553 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.478845 27523 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:10.522224 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 129206us
I20260709 12:18:10.488739 27457 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.529443 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:10.530639 27686 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
W20260709 12:18:10.531673 27703 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:18:10.534128 27638 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.534152 25796 tablet_copy_service.cc:342] P 955233c139344131a847044f17f8d348: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-7701d2ef98204ad0bedc9231fe829f69 received from {username='slave'} at 127.0.0.1:50194
I20260709 12:18:10.527130 25706 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:10.435153 27520 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.462445 27618 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:10.556900 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 425462us
I20260709 12:18:10.552909 25796 tablet_copy_service.cc:434] P 955233c139344131a847044f17f8d348: ending tablet copy session 10d0c7349085463595aedaf7c071e742-7701d2ef98204ad0bedc9231fe829f69 on tablet 7701d2ef98204ad0bedc9231fe829f69 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:18:10.535018 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:10.559891 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.382476 27690 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.566517 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 356) took 1401 ms. Trace:
I20260709 12:18:10.563402 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:10.528357 27695 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.564152 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.562345 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.565524 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.565723 25782 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:10.568106 26339 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:10.570847 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:10.566933 25631 rpcz_store.cc:276] 0709 12:18:09.161840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.162139 (+   299us) service_pool.cc:224] Handling call
0709 12:18:09.175008 (+ 12869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.263128 (+ 88120us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:09.298791 (+ 35663us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.299052 (+   261us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:09.299119 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:09.299188 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.562775 (+1263587us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b54001d65d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:10.562825 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.562944 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7064,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":1598,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":13210,"spinlock_wait_cycles":201859968}
I20260709 12:18:10.573261 25785 raft_consensus.cc:2368] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4.
I20260709 12:18:10.573724 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:10.577666 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.573014 27520 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:10.575366 27568 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:10.583041 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.583549 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.575343 27554 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:10.577488 27697 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:10.581059 27565 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.606231 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.556198 27488 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:10.578961 27562 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:10.603142 27680 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:10.582355 27435 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:18:10.613732 27696 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:18:10.609198 27697 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:10.621738 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.613461 27562 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:10.609975 27706 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:10.615223 27572 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.624866 26339 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:10.580353 27695 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.627705 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:10.637137 25631 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:18:10.618109 27554 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:10.630417 27708 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:18:10.631316 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1386) took 2349 ms. Trace:
I20260709 12:18:10.638990 25779 rpcz_store.cc:276] 0709 12:18:08.282185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.289403 (+  7218us) service_pool.cc:224] Handling call
0709 12:18:08.289713 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.259618 (+969905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.631068 (+1371450us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b5400257bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:10.631257 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160017792}
I20260709 12:18:10.631919 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.640611 25633 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 2: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:10.641583 27488 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:10.633296 27523 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:10.634652 27691 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:18:10.627151 27565 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.651924 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.651773 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:10.654525 25779 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:10.654948 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:10.655712 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.656535 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.659356 27554 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:10.659842 27678 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:10.660199 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:10.660475 27552 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:10.658432 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:10.657675 27706 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.668623 27435 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:10.669030 27435 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:10.669413 27435 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:10.660233 27678 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:10.675577 27678 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:10.683193 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.649981 27594 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.684679 25631 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:10.699352 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.700943 25782 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:10.687443 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.689044 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:10.690413 27587 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:10.676477 27554 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:10.690757 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:10.690815 27694 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.683303 27602 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:18:10.642973 27697 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.704063 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.710453 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:10.713209 25601 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:10.706259 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:10.696390 27666 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
I20260709 12:18:10.714538 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:10.701025 27581 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.003s
I20260709 12:18:10.718269 27641 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:18:10.711562 27611 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.004s
I20260709 12:18:10.694590 27523 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:18:10.719451 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.722779 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.723079 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.723613 25629 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:18:10.700730 27557 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.723862 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.731664 25636 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:10.727901 25631 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:10.728823 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:10.728735 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.734014 27641 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:10.708606 27618 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.709875 27569 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.718657 27666 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.724234 27552 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:10.729766 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:10.716486 26335 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:10.743357 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.715781 27656 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.733603 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.735950 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.722676 27706 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:10.741952 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:10.686542 27562 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.721248 27581 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.744916 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.769600 25636 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:10.731401 27479 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:10.771718 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:10.773272 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:10.783974 25779 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:10.771612 27474 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:18:10.729502 27611 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.788018 25783 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:10.747467 27712 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:10.735152 27641 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.730437 27554 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.771445 27598 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:10.792953 27710 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:18:10.738956 27557 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:10.795100 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 84712.7us
I20260709 12:18:10.782754 27440 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.799325 26342 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 2: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:10.786937 27695 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.788223 27602 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:10.788020 27569 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:18:10.794612 27712 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:10.824009 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50768.5us
I20260709 12:18:10.825374 27479 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:10.827039 27594 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:10.826577 27479 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.826577 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:10.863706 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:10.867197 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:10.870914 27656 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:10.822504 27712 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:10.807606 27440 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.884596 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:10.841370 27594 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:10.883721 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:10.895799 27648 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.901510 27440 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.879251 27611 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.798043 27598 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.886711 27645 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:10.905658 27627 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:10.932518 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.822906 27552 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.908360 27666 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.943329 27714 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:10.945026 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:10.953433 27581 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.956761 27598 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.957922 27435 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:10.961221 27435 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:10.962260 25638 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:10.970352 27675 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:10.970748 27587 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:10.970796 27675 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:10.981366 26307 leader_election.cc:400] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4.
I20260709 12:18:10.981953 26307 leader_election.cc:403] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:10.985692 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:10.971772 27675 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.987999 27675 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:10.993599 27651 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:10.995350 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1873) took 1520 ms. Trace:
I20260709 12:18:10.995872 25711 rpcz_store.cc:276] 0709 12:18:09.475234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.477230 (+  1996us) service_pool.cc:224] Handling call
0709 12:18:09.477495 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.486091 (+  8596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.995069 (+1508978us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b5400225bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:10.995276 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18317056}
I20260709 12:18:10.994446 27651 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:11.003988 27557 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:10.942142 27645 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.020520 27651 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:10.957685 27712 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.025372 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:11.023895 25629 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:11.028733 27645 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.028646 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 1871) took 1661 ms. Trace:
I20260709 12:18:11.038823 25633 raft_consensus.cc:2368] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4.
I20260709 12:18:11.038275 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.036949 25704 rpcz_store.cc:276] 0709 12:18:09.367099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.375872 (+  8773us) service_pool.cc:224] Handling call
0709 12:18:11.028453 (+1652581us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002ec7d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:11.028558 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747517312}
I20260709 12:18:11.033486 27678 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.035852 27695 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:11.066515 27660 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:11.050393 27624 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.072753 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.073364 25786 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:11.073310 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.074119 25785 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:11.073221 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:11.074668 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:11.068502 27660 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.075094 25782 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:11.154901 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:11.155280 27688 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:18:11.156208 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:11.115384 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.158811 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:11.075253 25779 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
W20260709 12:18:11.105566 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.115173 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:11.160950 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 86476.9us
I20260709 12:18:11.103516 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.161899 25711 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:11.162225 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:11.163161 25780 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:11.106907 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:11.068545 27690 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:11.177492 27520 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:11.153762 27567 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:11.185519 25782 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:11.164693 27488 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:11.066871 27695 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.187666 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:11.166682 27503 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:11.195365 25631 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:11.176597 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.196928 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:11.197714 25782 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.178234 27657 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:11.199393 27678 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.200491 27568 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.197019 27519 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:11.204854 27709 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.698s	user 0.002s	sys 0.000s
I20260709 12:18:11.206070 25782 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:11.207712 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:11.225353 25780 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.210181 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:11.213985 27662 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:11.216384 27660 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.218062 27695 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.220472 27562 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:11.221297 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 62559.8us
I20260709 12:18:11.188920 27712 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.226853 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:11.228256 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.240139 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.228500 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.228801 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.241813 25713 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:11.188014 27488 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.226039 27519 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.231495 25708 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:11.226847 27624 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:11.232683 26335 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:11.255247 27519 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:11.224568 25704 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:11.256731 27665 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.197058 27552 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:11.216570 25778 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.257843 27694 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.258177 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:50966 (request call id 1379) took 3061 ms. Trace:
I20260709 12:18:11.226145 27657 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.195430 27503 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.188203 27567 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.230537 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:11.234277 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.236430 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:11.235006 27662 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.224502 27568 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.251103 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.264649 25780 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:11.241110 27623 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:11.269560 27624 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:11.260324 27488 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.240562 27520 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:11.268802 27665 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.243173 27664 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.276957 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.267336 27503 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.295544 25711 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:11.282122 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.282814 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:11.296306 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.298570 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.302780 27618 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:11.303691 27695 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:11.305316 27666 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:11.297240 25706 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:11.306850 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.299907 27520 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.316882 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:11.258626 25778 rpcz_store.cc:276] 0709 12:18:08.196901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.206934 (+ 10033us) service_pool.cc:224] Handling call
0709 12:18:11.215095 (+3008161us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54000866d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5986a3d 00007f29ba956cd0 00007f29b3edb254 00007f29b3edb1e2 00007f29b3edb111 00007f29b3edb08c 00007f29b3eda342 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:11.258117 (+ 43022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257972992}
I20260709 12:18:11.321296 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:11.321930 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.322638 25707 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:11.285531 27629 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:18:11.327136 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.327809 25707 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 2: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:11.285125 27623 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.329480 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:11.334336 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.334407 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.335469 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.335417 25710 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:11.320884 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.309558 27611 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:18:11.307438 27664 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.400179 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:11.400502 25707 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.400978 26344 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:11.324276 25704 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:11.307636 27665 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:18:11.315850 26338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 395) took 2890 ms (client timeout 2128 ms). Trace:
W20260709 12:18:11.342825 27733 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:18:11.420110 27664 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.428702 27623 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.429942 27524 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.425312 27520 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.432301 27665 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:11.409811 26338 rpcz_store.cc:269] 0709 12:18:08.415399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.417663 (+  2264us) service_pool.cc:224] Handling call
0709 12:18:11.305757 (+2888094us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b54006b7e54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:11.305874 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2128261760}
I20260709 12:18:11.451491 27567 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.459319 27662 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.465466 25707 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:11.487912 27694 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4.
I20260709 12:18:11.492558 26343 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:11.487557 27488 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.495504 27553 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.495597 27488 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.499814 27457 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.508800 27440 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.509428 27440 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:11.512244 27568 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:11.514998 27737 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:18:11.515950 27503 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.517989 27503 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.518448 27440 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.519712 27440 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.522197 27567 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.522825 27567 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.523685 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:11.524775 27567 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.525209 25677 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:11.539590 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 309129us
I20260709 12:18:11.542743 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 368) took 1464 ms. Trace:
I20260709 12:18:11.543548 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:11.544251 25780 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.515959 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1941) took 2183 ms. Trace:
I20260709 12:18:11.541786 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:11.543299 25635 rpcz_store.cc:276] 0709 12:18:10.077923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.078330 (+   407us) service_pool.cc:224] Handling call
0709 12:18:10.099857 (+ 21527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.100163 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:10.329047 (+228884us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.424688 (+1095641us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b1800127ea0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a
0709 12:18:11.542364 (+117676us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:11.542433 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:11.542479 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.542572 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.542687 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117363,"prepare.queue_time_us":285807,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":1938,"raft.queue_time_us":118786,"spinlock_wait_cycles":27065088,"thread_start_us":351627,"threads_started":2,"wal-append.queue_time_us":182135}
I20260709 12:18:11.545668 25709 rpcz_store.cc:276] 0709 12:18:09.332162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.332370 (+   208us) service_pool.cc:224] Handling call
0709 12:18:09.332769 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.258335 (+925566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.515677 (+1257342us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b5400147154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:11.515885 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2088798080}
I20260709 12:18:11.551548 25637 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:11.553212 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.556202 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.557979 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.525876 27524 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:11.559880 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.525291 27567 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.565076 25780 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:11.565867 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.544348 27661 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:11.539590 27552 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:11.544142 27665 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:11.545025 27568 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.575229 25630 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:11.557497 27520 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:11.569140 27524 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.585613 25634 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:11.588152 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 1861) took 1407 ms (client timeout 1558 ms). Trace:
I20260709 12:18:11.588318 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.574635 27665 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.588034 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:11.588649 25783 rpcz_store.cc:269] 0709 12:18:10.180880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.181089 (+   209us) service_pool.cc:224] Handling call
0709 12:18:11.587949 (+1406860us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400098b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:11.588086 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331656448}
I20260709 12:18:11.588048 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.581043 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:11.596310 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:11.581338 27520 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:11.586616 27524 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:11.583463 27661 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:11.594750 27565 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:11.605779 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.595106 27665 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.607656 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.576761 27567 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.619338 27565 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.624338 27565 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.609706 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.602934 27554 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:11.611230 27678 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:11.626294 27554 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:11.598793 27520 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:11.627100 27565 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.627595 27565 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:11.603478 25778 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.634349 27453 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:11.594515 27712 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:11.635283 27453 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.611388 27503 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:11.612927 27661 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.627362 27554 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.603523 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.635881 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.625713 25706 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.594569 27552 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.651252 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:11.640132 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43797us
I20260709 12:18:11.659508 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.662840 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:11.676270 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:11.638253 27657 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:11.679625 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 83487.8us
I20260709 12:18:11.678220 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:11.688854 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:11.666200 27503 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:11.699730 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:11.638372 27712 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.680263 27668 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:11.708205 25706 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:11.685330 27729 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:11.719869 27697 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.013s	sys 0.031s
W20260709 12:18:11.721035 27697 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.013s	sys 0.031s
I20260709 12:18:11.720791 27620 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:11.722939 25534 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 300) took 5265 ms. Trace:
I20260709 12:18:11.693836 27453 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.695258 27618 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:11.692492 27662 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:18:11.727454 27624 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.696388 27657 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:18:11.724536 27620 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:11.696653 27522 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:18:11.732897 27668 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:11.735373 27618 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:11.756114 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67354.7us
I20260709 12:18:11.744467 27624 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:11.728375 25534 rpcz_store.cc:276] 0709 12:18:06.457167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.460582 (+  3415us) service_pool.cc:224] Handling call
0709 12:18:06.473163 (+ 12581us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:11.722847 (+5249684us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:07.631459 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:07.632105 (+   646us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:07.632152 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:18:07.632178 (+    26us) tablet.cc:662] Decoding operations
0709 12:18:07.632584 (+   406us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:07.632660 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:07.632701 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:18:07.632937 (+   236us) tablet.cc:701] Row locks acquired
0709 12:18:07.632957 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:18:07.633147 (+   190us) write_op.cc:270] Start()
0709 12:18:07.633282 (+   135us) write_op.cc:276] Timestamp: P: 1783599487633093 usec, L: 0
0709 12:18:07.633310 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:18:07.634093 (+   783us) log.cc:844] Serialized 599 byte log entry
0709 12:18:08.093119 (+459026us) op_driver.cc:464] REPLICATION: finished
0709 12:18:08.606354 (+513235us) write_op.cc:301] APPLY: starting
0709 12:18:08.607650 (+  1296us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:08.629857 (+ 22207us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:08.649899 (+ 20042us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:08.650217 (+   318us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:08.650420 (+   203us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:18:08.650478 (+    58us) write_op.cc:312] APPLY: finished
0709 12:18:08.650713 (+   235us) log.cc:844] Serialized 23 byte log entry
0709 12:18:08.654916 (+  4203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:08.655010 (+    94us) write_op.cc:454] Released schema lock
0709 12:18:08.655056 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:08.655444 (+   388us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:08.655468 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":695882,"reported_tablets":5,"spinlock_wait_cycles":11031808,"child_traces":[["op",{"apply.queue_time_us":484963,"apply.run_cpu_time_us":2245,"apply.run_wall_time_us":59142,"num_ops":1,"prepare.queue_time_us":172,"prepare.run_cpu_time_us":3019,"prepare.run_wall_time_us":14984,"replication_time_us":434026,"thread_start_us":368709,"threads_started":1}]]}
I20260709 12:18:11.713040 27565 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.751590 27620 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.749030 27747 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:11.755375 27618 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:11.755067 27657 consensus_queue.cc:1243] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:18:11.750658 27662 consensus_queue.cc:1243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:11.769955 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.777163 25629 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:11.779912 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:11.787165 25521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121020us
I20260709 12:18:11.791715 27641 raft_consensus.cc:1064] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
W20260709 12:18:11.810700 27746 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 12:18:11.812465 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.816295 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1758) took 1090 ms. Trace:
I20260709 12:18:11.816709 25633 rpcz_store.cc:276] 0709 12:18:10.725981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.733671 (+  7690us) service_pool.cc:224] Handling call
0709 12:18:11.816085 (+1082414us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400362754. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:11.816233 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619999488}
I20260709 12:18:11.818848 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:11.821784 27662 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.822414 27662 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.822826 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:11.823959 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:11.824851 27662 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.825022 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:11.825423 27662 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.784030 27657 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:11.826555 25632 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:18:11.826753 27657 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.827138 27657 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:11.827631 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.825925 27662 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.828433 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:11.828987 25713 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.830624 27662 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:11.830780 27553 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.830862 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.832163 25709 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.807940 27668 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.834659 27652 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:11.835042 27652 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:11.835412 27652 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.836262 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 403) took 1154 ms. Trace:
I20260709 12:18:11.839171 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:11.842221 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:11.845602 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.846257 25629 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:11.836787 26341 rpcz_store.cc:276] 0709 12:18:10.681824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.034220 (+352396us) service_pool.cc:224] Handling call
0709 12:18:11.035461 (+  1241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:11.036489 (+  1028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.036527 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:11.037040 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:18:11.703321 (+666281us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b18002b0ae0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:18:11.744667 (+ 41346us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:11.764105 (+ 19438us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.764324 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.764391 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.835914 (+ 71523us) raft_consensus.cc:1685] finished
0709 12:18:11.836040 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.836191 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16188,"prepare.queue_time_us":20933,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":2806,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":9579,"spinlock_wait_cycles":4956544,"thread_start_us":41158,"threads_started":1,"wal-append.queue_time_us":771159}
I20260709 12:18:11.847211 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:11.843495 27719 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.848166 27453 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:11.847636 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:11.848362 27712 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:11.849439 27690 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:11.851243 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.851696 27662 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:11.852442 27676 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:11.888118 27676 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:11.888857 27730 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.821s	user 0.007s	sys 0.000s
I20260709 12:18:11.888164 27641 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.890431 27676 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:11.890980 27453 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:11.808578 27745 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:18:11.855294 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.856590 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.870260 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:11.867584 27719 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:11.894927 27750 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:18:11.895099 27719 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.896499 27719 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.899804 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:11.900465 27665 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.900952 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:11.902271 25707 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:11.871932 25632 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:11.872457 27629 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:11.873276 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.761533 27503 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:11.874187 25638 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:11.875737 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:11.927867 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 170246us
I20260709 12:18:11.877996 25631 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:11.880555 25637 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:11.867529 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:11.799398 27611 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.881809 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:11.883383 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:11.886636 27690 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:11.931561 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49862.2us
W20260709 12:18:11.931864 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48573.1us
I20260709 12:18:11.932576 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:11.934741 27566 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:11.935734 27566 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:11.873951 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 569) took 1295 ms. Trace:
I20260709 12:18:11.937664 26336 rpcz_store.cc:276] 0709 12:18:10.579200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.579349 (+   149us) service_pool.cc:224] Handling call
0709 12:18:11.873074 (+1293725us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:11.873318 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.873542 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.873580 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:11.873632 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:11.873687 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.873756 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.873883 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20338944}
I20260709 12:18:11.885793 25709 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:18:11.924682 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.946703 27522 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:11.947651 27522 consensus_queue.cc:1243] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:18:11.948197 27522 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:11.949016 27522 consensus_queue.cc:1243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:18:11.952557 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76911.4us
I20260709 12:18:11.868387 27567 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:11.957530 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 88067.5us
I20260709 12:18:11.904559 27629 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.925454 27639 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.950085 27690 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:11.952347 27680 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:11.952636 27651 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:11.978156 25635 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:11.958279 27568 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:11.992731 25636 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:11.954936 27519 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:11.979965 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 76168.8us
W20260709 12:18:11.991060 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.972273 27629 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:12.031153 25629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:18:12.033103 25629 raft_consensus.cc:606] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership
I20260709 12:18:11.956252 27611 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.997617 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:11.998325 27568 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:12.069101 27568 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:12.034868 27676 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:12.029302 27668 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:12.035871 27651 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:11.992408 27524 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:11.977653 27639 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.095664 27639 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:12.093237 27524 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:12.097754 27553 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:12.098609 27618 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.102602 27624 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.103964 27624 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:12.106532 27618 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:12.108100 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:12.074666 27676 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.113538 27676 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:12.115258 25778 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:12.114670 27568 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.122812 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:12.124432 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 401) took 2051 ms. Trace:
I20260709 12:18:12.125712 26340 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:12.121142 27568 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:12.127767 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 562) took 1594 ms. Trace:
W20260709 12:18:12.129642 27520 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.002s
I20260709 12:18:12.045405 27716 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:18:12.126431 27562 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:12.124846 26342 rpcz_store.cc:276] 0709 12:18:10.072657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.072774 (+   117us) service_pool.cc:224] Handling call
0709 12:18:12.124216 (+2051442us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:12.124348 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599723904}
I20260709 12:18:12.133061 25713 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:12.142902 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 210372us
I20260709 12:18:12.147264 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:18:12.150506 25636 raft_consensus.cc:606] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership
I20260709 12:18:11.975052 27620 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:12.157581 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:12.091163 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 145292us
I20260709 12:18:12.159112 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:12.128156 26340 rpcz_store.cc:276] 0709 12:18:10.533357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.533484 (+   127us) service_pool.cc:224] Handling call
0709 12:18:12.125501 (+1592017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.126158 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.127573 (+  1415us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400563450. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:12.127706 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3660597120}
I20260709 12:18:12.170629 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:12.163057 26339 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:12.166972 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:12.171622 25709 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:12.167407 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:12.165287 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:12.097805 27474 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:18:12.188509 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:12.192999 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:12.146638 27651 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:12.196614 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:12.200501 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:12.203334 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:12.161111 27675 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.188721 27524 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:18:12.210301 25709 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:12.170802 27566 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:12.196573 27676 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:12.085934 27668 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.205168 27678 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:12.259965 27618 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:12.260414 27618 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:12.260782 27618 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:12.263648 27453 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:12.161917 27652 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.170627 27639 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:12.234439 25713 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:12.217412 27712 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:18:12.249784 27657 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.256887 27620 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:12.258978 27611 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:12.355195 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:12.193665 27562 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:12.162412 27520 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.226735 27615 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:12.367072 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:12.339818 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1891) took 1110 ms. Trace:
I20260709 12:18:12.341113 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:12.340999 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:12.371500 25712 rpcz_store.cc:276] 0709 12:18:11.229581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.243594 (+ 14013us) service_pool.cc:224] Handling call
0709 12:18:11.243931 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.315594 (+ 71663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.315626 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:11.315671 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:11.315700 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.339553 (+1023853us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b5400105754. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:12.339609 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.339758 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164314880}
I20260709 12:18:12.380432 25629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:51510
I20260709 12:18:12.217021 27675 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.389704 25634 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:12.389863 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:12.390453 25631 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:12.347569 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:12.230552 27567 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:12.381042 25629 raft_consensus.cc:606] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership
I20260709 12:18:12.361806 27474 tablet_bootstrap.cc:492] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:18:12.216886 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:12.388156 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:12.350714 27676 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:12.397260 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:12.377908 27729 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.644s	user 0.003s	sys 0.009s
I20260709 12:18:12.394475 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:12.409433 25636 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:18:12.417071 25631 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:12.411921 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 195153us
I20260709 12:18:12.413836 25634 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:12.413714 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:12.419781 25638 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:12.408645 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:12.420596 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:12.422705 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:12.424896 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:12.427378 27534 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.435352 25637 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:12.387230 27762 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:12.438457 25631 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:12.231478 27566 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:12.439471 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 477436us
I20260709 12:18:12.264282 27651 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.406154 27567 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:12.364337 27639 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.456074 27474 ts_tablet_manager.cc:1403] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 1.722s	user 0.040s	sys 0.016s
I20260709 12:18:12.429466 27729 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:12.381819 27690 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.380769 27615 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:12.542718 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:12.548101 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 207016us
I20260709 12:18:12.557862 27712 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:12.463093 27664 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:12.537107 27579 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:12.386914 27662 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:18:12.380051 27524 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 12:18:12.647189 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.451344 27534 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.664146 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:12.654886 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 232311us
W20260709 12:18:12.667970 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 108624us
I20260709 12:18:12.668885 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:12.580679 27519 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:12.660907 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 236143us
I20260709 12:18:12.674635 26341 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:12.683341 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:12.673787 27611 raft_consensus.cc:670] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:12.663254 27699 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:12.541299 27562 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.664141 27764 raft_consensus.cc:2764] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:18:12.738471 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:12.694818 27568 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.705045 27524 consensus_queue.cc:1243] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:12.583491 27676 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.699497 27668 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:12.730939 27657 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:12.731614 27694 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:12.708846 27773 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:12.776175 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 86669.7us
I20260709 12:18:12.695421 27579 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:18:12.686115 27519 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.738854 27520 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:12.755265 27762 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:12.704351 27785 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:18:12.756165 27699 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:12.733827 27567 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.776904 27675 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:12.929407 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 260620us
I20260709 12:18:12.751235 27662 consensus_queue.cc:1243] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:12.769586 27766 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:12.842335 27763 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 12:18:12.864074 27629 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.820s	user 0.011s	sys 0.000s
W20260709 12:18:12.928570 27757 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.411s	user 0.000s	sys 0.002s
I20260709 12:18:12.927479 27723 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:12.860179 27775 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:18:12.863327 27770 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:18:12.950783 27769 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.380s	user 0.001s	sys 0.001s
I20260709 12:18:12.871722 27712 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:12.974619 27565 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:12.919255 27594 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:12.999853 27594 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.002602 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 242) took 1068 ms. Trace:
W20260709 12:18:13.003875 27678 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.000s
I20260709 12:18:13.004387 27678 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:12.764564 27568 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.009717 25630 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:12.833983 27566 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:12.865021 27793 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:12.868319 27534 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:12.947901 27773 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:12.866604 27778 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:18:12.888945 27639 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:12.870950 27784 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.867362 27690 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:12.934131 27657 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:12.989709 27723 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.805402 27652 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:13.005229 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 241) took 1071 ms. Trace:
I20260709 12:18:12.952957 27694 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:12.970348 27572 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:13.003225 27712 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.014695 27660 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:13.126845 27660 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:13.128127 27712 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.127941 27660 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:13.129026 27660 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:13.002655 27699 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:13.033789 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:13.022658 27793 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.003216 25704 rpcz_store.cc:276] 0709 12:18:11.934431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.942608 (+  8177us) service_pool.cc:224] Handling call
0709 12:18:11.943065 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:11.943565 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.943610 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:11.944259 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.202751 (+258492us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:12.209798 (+  7047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.216435 (+  6637us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:13.002307 (+785872us) raft_consensus.cc:1685] finished
0709 12:18:13.002333 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.002527 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2883,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":135,"thread_start_us":246587,"threads_started":1,"wal-append.queue_time_us":491926}
I20260709 12:18:13.032619 27678 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.037230 27762 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:12.975363 27629 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.205333 27678 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:13.061579 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:13.064189 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:13.215054 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:13.068733 25636 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:18:13.085228 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.534s (SENT) [suppressed 14 similar messages]
I20260709 12:18:13.044093 25708 rpcz_store.cc:276] 0709 12:18:11.934050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.939775 (+  5725us) service_pool.cc:224] Handling call
0709 12:18:11.940331 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:11.941244 (+   913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.941287 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:11.941947 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.202401 (+260454us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:12.202524 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.202594 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:13.004923 (+802329us) raft_consensus.cc:1685] finished
0709 12:18:13.004954 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.005148 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":171,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":163,"thread_start_us":259974,"threads_started":1,"wal-append.queue_time_us":438375}
I20260709 12:18:12.976166 27567 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:13.109220 27572 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:13.044596 27657 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:13.115077 27594 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.044711 27786 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.120630 27723 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.042830 27784 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:13.123003 27773 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:13.124795 27618 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.864s	user 0.011s	sys 0.000s
I20260709 12:18:13.125609 27694 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:13.023902 27766 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:13.129917 27474 raft_consensus.cc:359] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:13.152472 27519 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:13.153441 27793 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:13.041814 27579 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:13.040710 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:13.274132 27795 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:13.203572 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1454) took 1312 ms. Trace:
I20260709 12:18:12.870507 27716 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.158s
W20260709 12:18:13.262723 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.007s	sys 0.002s
I20260709 12:18:13.199939 27565 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.211234 27794 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:13.289363 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 204178us
W20260709 12:18:13.451761 25676 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.650s (SENT)
W20260709 12:18:13.453289 25676 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.782s (SENT)
I20260709 12:18:13.308130 25636 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:13.454397 25636 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:13.319216 27690 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:13.333635 27641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.022s	sys 0.044s
W20260709 12:18:13.423048 25750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.000s	sys 0.043s
W20260709 12:18:13.468029 25750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.000s	sys 0.044s
W20260709 12:18:13.468506 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 768952us
I20260709 12:18:13.430622 25631 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:13.269719 27652 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.161s
I20260709 12:18:13.284327 27748 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:13.447060 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.008s	sys 0.002s
I20260709 12:18:13.436745 25632 rpcz_store.cc:276] 0709 12:18:11.891581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.904082 (+ 12501us) service_pool.cc:224] Handling call
0709 12:18:11.904411 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:11.951332 (+ 46921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.951377 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:11.955809 (+  4432us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.360398 (+404589us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:12.360456 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.360576 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:13.203328 (+842752us) raft_consensus.cc:1685] finished
0709 12:18:13.203348 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.203512 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20372,"raft.queue_time_us":21361,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":8633,"thread_start_us":380255,"threads_started":1,"wal-append.queue_time_us":1233035}
I20260709 12:18:13.427066 27474 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
W20260709 12:18:13.474565 27641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.022s	sys 0.045s
W20260709 12:18:13.500250 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.501104 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.455597 25636 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:13.206646 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.479317 25631 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:13.502938 25636 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:13.462241 25676 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.853s (SENT)
I20260709 12:18:13.423192 27594 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:13.423338 27503 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:13.415928 27618 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:13.510784 27782 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.308s	user 0.002s	sys 0.000s
I20260709 12:18:13.511834 27618 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.512516 26341 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:13.513182 27793 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:13.275414 27786 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:13.513935 27739 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 2.070s	user 0.011s	sys 0.001s
I20260709 12:18:13.514509 27519 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.515005 27519 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.515678 27786 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:13.505888 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43680us
W20260709 12:18:14.700502 25676 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.985s (SENT)
W20260709 12:18:13.437062 25602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.000s	sys 0.034s
W20260709 12:18:13.509929 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.693789 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:14.952335 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 1.25846e+06us
W20260709 12:18:13.559581 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 237369us
W20260709 12:18:13.633293 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.541157 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:14.962877 25778 raft_consensus.cc:2368] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3.
W20260709 12:18:14.963537 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1401) took 1444 ms (client timeout 1782 ms). Trace:
W20260709 12:18:14.963970 25778 rpcz_store.cc:269] 0709 12:18:13.518900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.522501 (+  3601us) service_pool.cc:224] Handling call
0709 12:18:14.963399 (+1440898us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:13.520834 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:13.596624 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:13.520679 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:13.519927 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:13.519011 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:13.521538 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:13.625525 25786 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:14.987124 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:14.983875 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:13.519673 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:15.039880 25533 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 4, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:15.041558 27835 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:15.042135 25787 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
W20260709 12:18:15.042663 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1402) took 1523 ms (client timeout 1853 ms). Trace:
I20260709 12:18:13.524197 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:13.592077 27809 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:18:13.683450 27811 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 12:18:15.043073 25787 rpcz_store.cc:269] 0709 12:18:13.519239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.519380 (+   141us) service_pool.cc:224] Handling call
0709 12:18:15.042564 (+1523184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.039840 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1407) took 1517 ms. Trace:
W20260709 12:18:14.432438 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.043102 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 56293.7us
W20260709 12:18:14.564940 25584 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:18:15.048013 25584 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 60051e5723b2429582407076a003096c already in progress
W20260709 12:18:15.048808 25584 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 60051e5723b2429582407076a003096c already in progress
I20260709 12:18:15.046908 25786 rpcz_store.cc:276] 0709 12:18:13.522766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.522895 (+   129us) service_pool.cc:224] Handling call
0709 12:18:13.604313 (+ 81418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.039220 (+1434907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.039609 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:14.586133 25705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 223) took 4787 ms (client timeout 1641 ms). Trace:
W20260709 12:18:14.676335 25635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.896s	user 0.048s	sys 0.077s
I20260709 12:18:14.584060 27821 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:15.050395 25635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.271s	user 0.048s	sys 0.078s
W20260709 12:18:15.050133 25705 rpcz_store.cc:269] 0709 12:18:09.798089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.819832 (+ 21743us) service_pool.cc:224] Handling call
0709 12:18:14.585555 (+4765723us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54003aaa54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:14.585860 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4022899072}
W20260709 12:18:14.683792 25707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 1899) took 2858 ms (client timeout 1502 ms). Trace:
W20260709 12:18:15.051692 25707 rpcz_store.cc:269] 0709 12:18:11.825275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.830581 (+  5306us) service_pool.cc:224] Handling call
0709 12:18:14.683224 (+2852643us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b54003c8cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:14.683547 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161875712}
I20260709 12:18:15.052109 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.052968 25705 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:15.053145 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.054311 25707 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:15.055233 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:14.706809 25636 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.203s	user 0.011s	sys 0.015s
I20260709 12:18:15.056272 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:14.734476 26338 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:15.057075 27699 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.057116 25603 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:15.057715 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.057925 27699 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.058852 25707 raft_consensus.cc:2410] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:18:15.059032 27723 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.059383 27754 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.060186 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.059984 27723 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.060750 27699 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.522596 27594 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:13.445259 27572 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.072846 27572 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:14.404845 27615 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:14.209434 27623 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:18:14.555709 27562 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.686s	user 0.015s	sys 0.004s
I20260709 12:18:14.403636 27665 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.632998 27694 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:14.320937 27664 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:15.085955 27664 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:15.087306 27664 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.669132 27678 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.479480 27690 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.090937 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.092199 27690 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.489506 27748 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.095657 27748 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:14.408881 27440 raft_consensus.cc:1064] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:18:13.542508 27474 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:15.101007 27664 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:13.523379 27503 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.105819 27503 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:13.517191 27519 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:14.289089 27756 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 2.237s	user 0.005s	sys 0.014s
I20260709 12:18:13.466004 27565 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.111816 25632 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:18:15.113873 27565 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.524395 27568 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:13.458422 27579 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:14.692162 27618 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:15.120375 27618 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:14.604256 27624 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:15.123235 27624 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:15.124547 27624 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:13.488809 27629 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:13.540617 27641 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:15.129452 27641 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:13.706947 27657 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:14.404896 27760 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 2.147s	user 0.010s	sys 0.017s
I20260709 12:18:13.534852 27762 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.134805 27762 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.137411 27762 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:13.428474 27766 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:15.141534 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.666702 27767 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.143714 27767 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:14.607426 27776 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.753s	user 0.006s	sys 0.010s
I20260709 12:18:13.684337 27784 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:14.521620 27790 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.650s	user 0.003s	sys 0.012s
I20260709 12:18:14.607342 27792 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:15.150084 27792 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:15.151397 27792 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:13.476615 27794 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:14.776768 27788 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.320s	user 0.007s	sys 0.007s
I20260709 12:18:15.156668 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 588) took 1672 ms. Trace:
I20260709 12:18:14.667235 27795 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.158460 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1908) took 2885 ms. Trace:
W20260709 12:18:14.787626 25602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.381s	user 0.001s	sys 0.037s
I20260709 12:18:14.784153 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1765) took 2400 ms. Trace:
I20260709 12:18:14.806435 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.159623 25630 rpcz_store.cc:276] 0709 12:18:12.383905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.384065 (+   160us) service_pool.cc:224] Handling call
0709 12:18:12.435512 (+ 51447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.010375 (+574863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.783320 (+1772945us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54001c5cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:14.783843 (+   523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311074688}
I20260709 12:18:15.158957 25712 rpcz_store.cc:276] 0709 12:18:12.272776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.373068 (+100292us) service_pool.cc:224] Handling call
0709 12:18:12.373366 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.373609 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.373632 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:12.374111 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.672439 (+298328us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:12.672484 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.672555 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.775564 (+2103009us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:15.157751 (+382187us) raft_consensus.cc:1685] finished
0709 12:18:15.157788 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.158220 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30523264,"thread_start_us":235397,"threads_started":1,"wal-append.queue_time_us":859412}
I20260709 12:18:14.816655 26335 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:18:14.818935 27826 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.159047 27795 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.166296 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 589) took 1681 ms. Trace:
I20260709 12:18:15.166296 27826 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.166903 26335 rpcz_store.cc:276] 0709 12:18:13.484779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.488020 (+  3241us) service_pool.cc:224] Handling call
0709 12:18:14.814137 (+1326117us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aeee 000000000036eb39 00007f29ba8de1e0 00007f29ba9a70c4 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:14.814779 (+   642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.165916 (+351137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.166217 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34439552}
I20260709 12:18:14.823592 27822 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:14.827867 27828 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:14.828572 25634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.363s	user 0.026s	sys 0.073s
I20260709 12:18:14.830775 27830 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.168874 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:15.169492 25634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.721s	user 0.027s	sys 0.074s
I20260709 12:18:14.851478 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 585) took 2174 ms. Trace:
I20260709 12:18:14.855096 27768 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:15.168975 27826 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.169744 27830 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.170470 27768 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.170809 27828 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.171777 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.172621 25712 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.172745 27830 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:14.857421 27829 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.173190 27768 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:14.701007 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:15.176646 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 476439us
I20260709 12:18:15.176800 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1764) took 2793 ms. Trace:
I20260709 12:18:14.860972 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:14.862215 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:14.864459 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.178304 25713 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:15.178519 25704 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:15.179037 27766 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:14.869004 27729 raft_consensus.cc:993] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:18:14.876307 27832 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.179456 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.180079 27832 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.180812 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.181728 27519 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:14.879118 25709 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:18:15.173748 27829 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:15.183725 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.185014 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.185153 27829 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:14.881666 27457 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.186383 25710 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.186753 27457 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.187449 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.188313 25710 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:15.185801 27829 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:14.882870 27764 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:14.884281 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:14.881753 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 245) took 2932 ms. Trace:
I20260709 12:18:15.157151 26339 rpcz_store.cc:276] 0709 12:18:13.484328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.484789 (+   461us) service_pool.cc:224] Handling call
0709 12:18:13.485098 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.118755 (+1633657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:15.118798 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:15.118871 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:15.118899 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.156283 (+ 37384us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54004975d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:15.156326 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.156490 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3756205312}
I20260709 12:18:15.182199 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.192428 25713 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.191248 25711 rpcz_store.cc:276] 0709 12:18:11.950269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.950676 (+   407us) service_pool.cc:224] Handling call
0709 12:18:11.951147 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:11.951617 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.951657 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:11.952261 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.331757 (+379496us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:12.669815 (+338058us) log.cc:844] Serialized 14 byte log entry
0709 12:18:14.880524 (+2210709us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b1800063e20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287
0709 12:18:14.880852 (+   328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.881000 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.881085 (+    85us) raft_consensus.cc:1685] finished
0709 12:18:14.881210 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.881509 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55324,"prepare.queue_time_us":122181,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":105619,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":18978,"spinlock_wait_cycles":40201088,"thread_start_us":720065,"threads_started":3,"wal-append.queue_time_us":695080,"wal-append.run_cpu_time_us":9263,"wal-append.run_wall_time_us":1623651}
I20260709 12:18:14.910420 27675 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.911314 27785 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:14.927322 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45385.2us
I20260709 12:18:14.966956 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1405) took 1446 ms. Trace:
I20260709 12:18:14.969041 25781 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:14.969287 27719 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:15.035545 25784 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:14.972678 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.035656 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1406) took 1515 ms (client timeout 1669 ms). Trace:
W20260709 12:18:15.040337 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1413) took 1350 ms (client timeout 1534 ms). Trace:
I20260709 12:18:15.040833 25783 raft_consensus.cc:2368] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:15.048084 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:15.051062 27755 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:14.691934 27823 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.054699 27774 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:14.692836 27824 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.056347 27661 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.060453 27754 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.061220 25707 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:15.047605 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.061933 27773 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:14.734987 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:14.304960 27534 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.007s	sys 0.003s
I20260709 12:18:15.058837 27552 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:15.062831 27620 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:18:15.063591 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.058796 27680 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:15.064819 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:15.221228 25785 rpcz_store.cc:269] 0709 12:18:13.520342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.521245 (+   903us) service_pool.cc:224] Handling call
0709 12:18:15.035478 (+1514233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.222347 27719 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:15.057397 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 403) took 1750 ms. Trace:
I20260709 12:18:15.223117 27719 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:15.222882 27824 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.223476 25707 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:15.224287 26344 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:15.223985 27823 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.224526 27719 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:15.066926 27821 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.226208 27824 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.199152 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 396) took 2134 ms. Trace:
I20260709 12:18:15.081656 27594 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:15.098402 27440 raft_consensus.cc:1068] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Unable to promote non-voter 10d0c7349085463595aedaf7c071e742: Service unavailable: leader transfer in progress
W20260709 12:18:15.081291 27562 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.212s	user 0.019s	sys 0.004s
W20260709 12:18:15.222455 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 175134us
I20260709 12:18:15.127128 25533 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:15.228498 27821 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.078451 27623 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:15.153352 27792 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.227476 25634 rpcz_store.cc:276] 0709 12:18:13.065063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.065191 (+   128us) service_pool.cc:224] Handling call
0709 12:18:13.065498 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.065764 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:13.447566 (+381802us) log.cc:844] Serialized 14 byte log entry
0709 12:18:15.198702 (+1751136us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:15.198796 (+    94us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:15.198858 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.198978 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.199100 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28624,"prepare.queue_time_us":127630,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":34260,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":2008974,"threads_started":2,"wal-append.queue_time_us":253798,"wal-append.run_cpu_time_us":1826,"wal-append.run_wall_time_us":1011677}
I20260709 12:18:15.217233 27784 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.228850 25533 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 4, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:13.513756 27793 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.234869 27785 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.221462 27773 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.237075 27773 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.237339 27793 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:15.196221 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 587) took 2516 ms. Trace:
I20260709 12:18:15.155601 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 591) took 1670 ms. Trace:
I20260709 12:18:15.160749 25779 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
W20260709 12:18:15.238937 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1415) took 1548 ms (client timeout 1513 ms). Trace:
W20260709 12:18:15.239629 25779 rpcz_store.cc:269] 0709 12:18:13.690579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.805421 (+1114842us) service_pool.cc:224] Handling call
0709 12:18:15.238837 (+433416us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.240175 26342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 586) took 2561 ms (client timeout 1752 ms). Trace:
I20260709 12:18:15.240799 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.241231 27754 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.194394 26343 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:14.815057 27825 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.242522 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.194347 27675 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.243314 27825 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.215696 27832 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:14.819149 27827 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.251443 27623 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.251922 27827 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.253235 27623 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.168807 27822 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.254704 27474 ts_tablet_manager.cc:1434] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 2.679s	user 0.022s	sys 0.009s
I20260709 12:18:15.201423 27678 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:14.843948 27831 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.173609 27828 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.173918 26337 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:15.174623 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.223383 25636 rpcz_store.cc:276] 0709 12:18:13.307355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.307552 (+   197us) service_pool.cc:224] Handling call
0709 12:18:13.307970 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.056931 (+1748961us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:15.056997 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:15.057070 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:15.057104 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.057170 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.057324 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4575,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":217}
I20260709 12:18:15.237963 26336 rpcz_store.cc:276] 0709 12:18:12.679597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.679759 (+   162us) service_pool.cc:224] Handling call
0709 12:18:14.934188 (+2254429us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:14.934562 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.194917 (+260355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:15.194959 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:15.195035 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:15.195079 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.195810 (+   731us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540036ae54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:15.195871 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.196027 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604909568}
I20260709 12:18:15.237440 25637 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:18:14.867540 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.126343 27624 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.221303 25780 rpcz_store.cc:276] 0709 12:18:13.520079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.520606 (+   527us) service_pool.cc:224] Handling call
0709 12:18:14.966737 (+1446131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.067050 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.226133 25707 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:15.216558 27457 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.228585 25751 leader_election.cc:400] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4.
I20260709 12:18:15.269235 25751 leader_election.cc:403] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:15.230271 27784 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:15.199994 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.219038 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:15.146248 27767 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.230835 27552 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:15.221630 25784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1400) took 1703 ms (client timeout 1650 ms). Trace:
W20260709 12:18:15.232297 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1404) took 1712 ms (client timeout 1985 ms). Trace:
I20260709 12:18:15.233196 27680 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:15.279037 25629 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:15.279187 25786 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
W20260709 12:18:15.279843 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1425) took 1587 ms (client timeout 1702 ms). Trace:
I20260709 12:18:15.279574 27678 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.237396 27785 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.280537 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:15.281247 25629 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:15.281878 25676 connection.cc:256] RPC call timeout handler was delayed by -1.6924393399999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
I20260709 12:18:15.233822 27755 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.284154 27755 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.176939 27795 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:15.274796 25783 rpcz_store.cc:269] 0709 12:18:13.519808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.520358 (+   550us) service_pool.cc:224] Handling call
0709 12:18:15.232196 (+1711838us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.274380 25784 rpcz_store.cc:269] 0709 12:18:13.518480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.518690 (+   210us) service_pool.cc:224] Handling call
0709 12:18:15.221500 (+1702810us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.280310 25786 rpcz_store.cc:269] 0709 12:18:13.692360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.065888 (+1373528us) service_pool.cc:224] Handling call
0709 12:18:15.279746 (+213858us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.231912 25533 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:15.036664 25782 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:15.239652 25705 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:15.204775 27565 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.198185 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.245813 27675 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.305850 27675 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4.
I20260709 12:18:15.161935 27794 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:15.165143 27579 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:15.241622 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1420) took 1550 ms (client timeout 1751 ms). Trace:
I20260709 12:18:15.246668 25710 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
W20260709 12:18:15.240569 26342 rpcz_store.cc:269] 0709 12:18:12.678651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.678868 (+   217us) service_pool.cc:224] Handling call
0709 12:18:14.734639 (+2055771us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:15.240070 (+505431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54889856}
I20260709 12:18:15.248237 25787 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:15.315092 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1411) took 1625 ms (client timeout 1569 ms). Trace:
W20260709 12:18:15.315604 25787 rpcz_store.cc:269] 0709 12:18:13.689560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.046550 (+1356990us) service_pool.cc:224] Handling call
0709 12:18:15.314984 (+268434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.248116 25713 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:15.217979 27488 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.199008 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.199810 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:15.245813 27825 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:15.249933 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1408) took 1726 ms (client timeout 2136 ms). Trace:
W20260709 12:18:15.242226 25778 rpcz_store.cc:269] 0709 12:18:13.690112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.983318 (+1293206us) service_pool.cc:224] Handling call
0709 12:18:15.040238 (+ 56920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.218385 27554 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.250885 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.164451 27641 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.173679 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:15.339946 25781 rpcz_store.cc:269] 0709 12:18:13.523105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.537330 (+ 14225us) service_pool.cc:224] Handling call
0709 12:18:15.249842 (+1712512us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.197398 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:15.121668 27618 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.351465 27618 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.174945 26341 rpcz_store.cc:276] 0709 12:18:12.677065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.677259 (+   194us) service_pool.cc:224] Handling call
0709 12:18:12.677668 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.513101 (+835433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.850615 (+1337514us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b540063fe54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:14.851212 (+   597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919299840}
I20260709 12:18:15.066118 27723 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:15.255187 27474 ts_tablet_manager.cc:1467] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Tablet startup took 2688ms
I20260709 12:18:15.268836 25708 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:18:15.223052 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.177332 25635 rpcz_store.cc:276] 0709 12:18:12.383448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.384424 (+   976us) service_pool.cc:224] Handling call
0709 12:18:12.384714 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.406243 (+ 21529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.406276 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:12.416171 (+  9895us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.670250 (+254079us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:12.779446 (+109196us) log.cc:844] Serialized 14 byte log entry
0709 12:18:14.520812 (+1741366us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:15.051134 (+530322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.051306 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.176394 (+125088us) raft_consensus.cc:1685] finished
0709 12:18:15.176519 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.176737 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54273,"prepare.queue_time_us":376772,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":51,"raft.queue_time_us":164648,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":29810,"spinlock_wait_cycles":5888128,"thread_start_us":2490132,"threads_started":3,"wal-append.queue_time_us":343553}
I20260709 12:18:15.269863 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.271296 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:15.266026 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.361330 26343 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:15.271481 27773 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.282039 27678 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:15.291173 25676 connection.cc:256] RPC call timeout handler was delayed by -1.6744393399999353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
I20260709 12:18:15.205338 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.287357 27680 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.293314 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.366310 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.295307 25709 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:15.363528 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.296002 27822 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.296505 27552 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.269315 27827 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.297052 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1403) took 1777 ms. Trace:
I20260709 12:18:15.297839 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:15.368673 25676 connection.cc:256] RPC call timeout handler was delayed by -1.55555176200005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
I20260709 12:18:15.298677 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.300676 27784 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.305588 27620 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:15.237819 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 592) took 1745 ms. Trace:
I20260709 12:18:15.368673 25782 rpcz_store.cc:276] 0709 12:18:13.519557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.519676 (+   119us) service_pool.cc:224] Handling call
0709 12:18:15.296965 (+1777289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.370419 26344 rpcz_store.cc:276] 0709 12:18:13.492938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.493152 (+   214us) service_pool.cc:224] Handling call
0709 12:18:14.931811 (+1438659us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:14.932267 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.224717 (+292450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.237562 (+ 12845us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:15.237757 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705289856}
I20260709 12:18:15.371182 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:15.307531 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 79252.9us
I20260709 12:18:15.372598 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.372643 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.373940 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:15.314098 25779 rpcz_store.cc:269] 0709 12:18:13.691532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.240400 (+1548868us) service_pool.cc:224] Handling call
0709 12:18:15.241537 (+  1137us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.374782 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1423) took 1682 ms (client timeout 2006 ms). Trace:
I20260709 12:18:15.374938 25784 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:15.375377 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.375746 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:15.376408 25786 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:15.209331 27729 raft_consensus.cc:1081] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
W20260709 12:18:15.324199 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.311444 27565 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.319768 25533 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:15.379431 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1421) took 1687 ms (client timeout 2073 ms). Trace:
I20260709 12:18:15.238101 26340 rpcz_store.cc:276] 0709 12:18:13.485371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.487126 (+  1755us) service_pool.cc:224] Handling call
0709 12:18:14.665474 (+1178348us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:14.666166 (+   692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.142434 (+476268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:15.142499 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:15.142579 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:15.142636 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.155035 (+ 12399us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54000a5d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:15.155111 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.155338 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1162481280}
I20260709 12:18:15.327127 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.331444 25632 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:15.332134 25638 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:15.332863 27488 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.335569 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.221309 27661 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.338907 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.344111 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.338912 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.345693 25629 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:15.382119 25780 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:15.294730 27748 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.348773 25712 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:15.349443 25711 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.349552 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:15.350054 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 152753us
I20260709 12:18:15.361398 25634 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:15.271057 27774 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:15.362162 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.392946 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:15.387748 25780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1412) took 1697 ms (client timeout 1785 ms). Trace:
W20260709 12:18:15.393998 25780 rpcz_store.cc:269] 0709 12:18:13.689821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.309200 (+1619379us) service_pool.cc:224] Handling call
0709 12:18:15.387698 (+ 78498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.376299 25779 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
W20260709 12:18:15.324899 27534 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.846s	user 0.010s	sys 0.003s
W20260709 12:18:15.375442 25784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1417) took 1684 ms (client timeout 2000 ms). Trace:
I20260709 12:18:15.376417 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:15.396706 25784 rpcz_store.cc:269] 0709 12:18:13.691013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.292914 (+1601901us) service_pool.cc:224] Handling call
0709 12:18:15.375386 (+ 82472us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.375227 25783 rpcz_store.cc:269] 0709 12:18:13.692023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.297498 (+1605475us) service_pool.cc:224] Handling call
0709 12:18:15.374708 (+ 77210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.369812 25707 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
W20260709 12:18:15.379891 25787 rpcz_store.cc:269] 0709 12:18:13.691682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.316461 (+1624779us) service_pool.cc:224] Handling call
0709 12:18:15.379361 (+ 62900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.381330 25782 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:15.290727 27831 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.381846 25630 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:15.390535 25713 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 2: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:15.386778 25785 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:15.387176 26339 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:15.388401 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.386929 25633 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:15.397069 25635 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
W20260709 12:18:15.401412 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1426) took 1708 ms (client timeout 1722 ms). Trace:
W20260709 12:18:15.402215 25785 rpcz_store.cc:269] 0709 12:18:13.692571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.222616 (+1530045us) service_pool.cc:224] Handling call
0709 12:18:15.401355 (+178739us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.387421 27641 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:15.398133 25631 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:15.409770 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.413610 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.401919 25533 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:15.428498 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1416) took 1737 ms (client timeout 2145 ms). Trace:
W20260709 12:18:15.429090 25781 rpcz_store.cc:269] 0709 12:18:13.690838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.347104 (+1656266us) service_pool.cc:224] Handling call
0709 12:18:15.428367 (+ 81263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.432147 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.432330 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:15.432250 25676 connection.cc:256] RPC call timeout handler was delayed by -1.5501517620000413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 12:18:15.386343 26337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 590) took 1901 ms (client timeout 1912 ms). Trace:
I20260709 12:18:15.429196 25629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
W20260709 12:18:15.436749 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.437121 27752 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:15.437707 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:15.438110 25638 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:15.438292 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 281309us
I20260709 12:18:15.438897 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.442523 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:15.443560 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:15.484483 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 69931.2us
I20260709 12:18:15.413405 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.443921 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.452843 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1419) took 1761 ms. Trace:
I20260709 12:18:15.520855 25783 rpcz_store.cc:276] 0709 12:18:13.691372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.399988 (+1708616us) service_pool.cc:224] Handling call
0709 12:18:15.452783 (+ 52795us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.472560 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.473225 27831 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.474231 27565 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.476890 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:15.477715 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.382586 27554 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.482793 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:15.446199 26337 rpcz_store.cc:269] 0709 12:18:13.485069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.487654 (+  2585us) service_pool.cc:224] Handling call
0709 12:18:15.384163 (+1896509us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54005a0354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:15.386254 (+  2091us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296953728}
I20260709 12:18:15.487589 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.493484 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.530020 26336 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 1.
W20260709 12:18:15.535596 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.529031 25787 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:15.535918 25706 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.494679 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.484902 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.542667 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:15.543570 25712 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:15.545030 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.495064 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.508013 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.508625 27488 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:15.494689 25778 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:15.510162 27661 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.511018 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:15.511396 25676 connection.cc:256] RPC call timeout handler was delayed by -1.4898517619999438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 12:18:15.364188 27474 ts_tablet_manager.cc:1469] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Trace:
0709 12:18:02.345072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.345887 (+   815us) service_pool.cc:224] Handling call
0709 12:18:04.457995 (+2112108us) ts_tablet_manager.cc:934] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
0709 12:18:07.202113 (+2744118us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:07.716291 (+514178us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:07.716403 (+   112us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:08.352420 (+636017us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:08.352867 (+   447us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:08.353299 (+   432us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:08.353442 (+   143us) inbound_call.cc:177] Queueing success response
0709 12:18:10.729415 (+2375973us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:18:12.576054 (+1846639us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:18:13.003508 (+427454us) tablet_replica.cc:261] Starting instrumentation
0709 12:18:13.004202 (+   694us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":336,"mutex_wait_us":13971,"raft.queue_time_us":13994,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":228,"tablet-copy.queue_time_us":1113751,"tablet-download-pool.queue_time_us":59094,"tablet-download-pool.run_cpu_time_us":2288,"tablet-download-pool.run_wall_time_us":1072679,"thread_start_us":1075891,"threads_started":2}
I20260709 12:18:15.468878 27752 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:15.516973 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.556365 25676 connection.cc:256] RPC call timeout handler was delayed by -1.4506093699999383s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27s for a response
I20260709 12:18:15.518847 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:15.556744 25676 connection.cc:256] RPC call timeout handler was delayed by -1.4250517620000664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
I20260709 12:18:15.496572 25636 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:15.557057 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3503517619999457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 12:18:15.520452 25710 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.448761 25629 raft_consensus.cc:606] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.488737 25533 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:15.413551 27748 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:15.529424 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121595us
I20260709 12:18:15.532291 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.540807 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.536963 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.563700 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.563794 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:15.564715 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:15.564925 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.565824 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.566080 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.566408 25636 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:15.567229 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.567425 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.567740 25636 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:15.568595 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.569253 25636 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:15.569856 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.570694 27661 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:15.571063 27488 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:15.571100 27661 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.571825 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.572628 26342 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.573354 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.573694 27748 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.574369 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:15.577277 25637 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
W20260709 12:18:15.542673 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.532807 25750 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:15.549337 25785 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:18:15.547381 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:15.551616 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.571856 27661 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.579377 25783 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:15.553470 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.561988 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:15.562012 27752 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.548708 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:15.579466 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46766.6us
I20260709 12:18:15.527940 26335 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:15.580188 25632 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:15.536288 27554 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.571144 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.493573 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.605315 26342 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:15.605675 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.588215 27793 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:15.590423 25638 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:15.614322 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.606869 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.609915 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:15.615983 25634 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:15.613704 25784 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.617758 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.617758 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:15.617864 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.618744 25636 raft_consensus.cc:2376] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7. Re-sending same reply.
I20260709 12:18:15.619187 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:15.619324 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.620246 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.620364 25787 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:15.622138 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:15.618584 25783 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:15.622965 25632 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:15.623108 25786 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:15.623955 25781 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.624150 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.588732 27831 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:18:15.575484 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1410) took 1960 ms (client timeout 2246 ms). Trace:
W20260709 12:18:15.632652 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1418) took 1941 ms (client timeout 2228 ms). Trace:
W20260709 12:18:15.633116 25786 rpcz_store.cc:269] 0709 12:18:13.691201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.369476 (+1678275us) service_pool.cc:224] Handling call
0709 12:18:15.632590 (+263114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.634205 25630 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:15.614327 27793 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:15.639694 27793 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:15.605819 26338 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:15.624185 27624 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:15.642802 27624 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.550145 27794 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:18:15.633955 26337 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 2: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:15.634763 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:15.645988 25785 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:15.643203 27624 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.650362 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:15.651150 25750 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.649158 25631 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
W20260709 12:18:15.652653 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 71205.8us
I20260709 12:18:15.654089 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.654956 25712 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.646625 25533 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:15.659703 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 48222.7us
I20260709 12:18:15.660423 25783 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:15.661079 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.664732 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1409) took 2137 ms. Trace:
W20260709 12:18:15.665179 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.665267 25783 rpcz_store.cc:276] 0709 12:18:13.523420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.636660 (+2113240us) service_pool.cc:224] Handling call
0709 12:18:15.647661 (+ 11001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.660800 (+ 13139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.660994 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.656374 27615 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:15.669346 27615 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.669600 25784 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:15.669795 25787 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:15.669878 27615 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.670265 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1427) took 1977 ms. Trace:
I20260709 12:18:15.644439 27675 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.670778 25784 rpcz_store.cc:276] 0709 12:18:13.692720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.660153 (+1967433us) service_pool.cc:224] Handling call
0709 12:18:15.669272 (+  9119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.670035 (+   763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.670193 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.651921 27554 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:15.668880 27568 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:15.670393 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1422) took 1978 ms. Trace:
I20260709 12:18:15.676775 25787 rpcz_store.cc:276] 0709 12:18:13.691848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.643318 (+1951470us) service_pool.cc:224] Handling call
0709 12:18:15.669631 (+ 26313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.670193 (+   562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.670348 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.679239 25786 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:15.680027 25629 pending_rounds.cc:85] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 4
I20260709 12:18:15.680040 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1424) took 1987 ms. Trace:
I20260709 12:18:15.680790 25629 pending_rounds.cc:107] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:18:15.681339 25629 raft_consensus.cc:2889] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:15.680972 25786 rpcz_store.cc:276] 0709 12:18:13.692183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.652086 (+1959903us) service_pool.cc:224] Handling call
0709 12:18:15.678906 (+ 26820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.679752 (+   846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.679976 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.682914 27534 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:15.684542 25786 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:15.688469 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:15.689726 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:15.650696 27831 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:15.712286 27831 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:15.634785 25778 rpcz_store.cc:269] 0709 12:18:13.615525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.379927 (+1764402us) service_pool.cc:224] Handling call
0709 12:18:15.554421 (+174494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.675450 27519 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:15.747545 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:15.754242 25712 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:15.674746 27828 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:15.672240 26308 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:15.674127 25533 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 3 to 6, term changed from 3 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to <none>, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 4 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:18:15.772341 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.773732 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:15.775231 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 126760us
W20260709 12:18:15.775386 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 108053us
I20260709 12:18:15.759853 27748 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.720305 27568 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:15.774803 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 206286us
I20260709 12:18:15.761221 27519 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.764019 27829 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:15.764038 27828 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.797286 25784 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:15.803937 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1429) took 2110 ms. Trace:
I20260709 12:18:15.804474 25784 rpcz_store.cc:276] 0709 12:18:13.693042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.758659 (+2065617us) service_pool.cc:224] Handling call
0709 12:18:15.797104 (+ 38445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.803608 (+  6504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.803864 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.807071 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.790084 27824 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:15.789436 27795 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:15.778367 27821 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:15.793004 27568 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:15.793385 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:15.795234 27519 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:15.780366 26365 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1060 after lost signal to thread 24424
I20260709 12:18:15.797909 27829 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.822640 25783 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:15.775604 27675 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:15.828999 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:15.818457 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:15.830338 25712 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.825222 25784 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.836242 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1428) took 2130 ms. Trace:
I20260709 12:18:15.836663 25783 rpcz_store.cc:276] 0709 12:18:13.692874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.759358 (+2066484us) service_pool.cc:224] Handling call
0709 12:18:15.799651 (+ 40293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.823220 (+ 23569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.823473 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.810444 25778 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.847563 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-ef4fd24cfa184aa8a8a1de8f32296e6a received from {username='slave'} at 127.0.0.1:60736
I20260709 12:18:15.814597 27457 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.802237 27828 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:15.815029 27824 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.834609 25533 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:15.851341 27554 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:15.829000 27829 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:15.852881 25601 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:15.863646 25781 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:15.866729 25783 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:15.819773 27821 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:15.863263 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-ef4fd24cfa184aa8a8a1de8f32296e6a on tablet ef4fd24cfa184aa8a8a1de8f32296e6a with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:18:15.860633 27824 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.860783 27766 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:18:15.868371 26365 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1740 after lost signal to thread 25570
I20260709 12:18:15.863651 27554 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:15.890272 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45584.7us
I20260709 12:18:15.893065 25783 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:15.833777 27748 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:15.882247 25533 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:15.903438 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 242345us
I20260709 12:18:15.877102 27821 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:15.887815 27488 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:15.889356 27766 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.926353 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:15.927811 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:15.900795 27554 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:15.955381 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:15.956257 25709 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:15.933243 25533 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:15.897181 27457 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:15.971900 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.979861 25533 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 5, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 5 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:15.932559 27825 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.965451 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48692.4us
I20260709 12:18:15.930779 27766 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.983400 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:15.985617 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:15.992744 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78095.1us
I20260709 12:18:15.954859 27565 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:15.997479 27565 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:15.997977 27565 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:15.998297 27840 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:18:16.000234 27457 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:16.000718 27846 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:18:16.009979 27565 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.011636 27748 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.012039 27675 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:16.012372 27748 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:18:16.012938 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.015554 27565 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:15.989552 27488 raft_consensus.cc:2764] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:16.012908 27748 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:16.026945 27748 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.965090 27534 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:16.036692 25712 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:16.038419 25533 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:16.040967 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:15.991868 26365 debug-util.cc:398] Leaking SignalData structure 0x7b0800373f80 after lost signal to thread 25662
I20260709 12:18:15.926970 27661 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.052397 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:16.059036 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.059973 25712 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
W20260709 12:18:16.064491 27847 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:18:16.065160 27754 raft_consensus.cc:1064] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:18:16.066882 27795 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:16.067269 27752 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.056067 26337 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:18:16.042090 27825 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.067341 27795 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:16.064541 26365 debug-util.cc:398] Leaking SignalData structure 0x7b080035b4a0 after lost signal to thread 25737
I20260709 12:18:16.078460 27661 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.106576 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:16.106652 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76977us
I20260709 12:18:16.067727 27754 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.074031 27852 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:16.113391 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 127926us
I20260709 12:18:16.114909 25712 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 1.
I20260709 12:18:16.121003 25603 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.126612 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.132376 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:16.118275 27831 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.121778 27623 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.130564 26365 debug-util.cc:398] Leaking SignalData structure 0x7b08002a5180 after lost signal to thread 25811
I20260709 12:18:16.142789 27747 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.143255 27747 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:16.143621 27747 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.143750 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:16.145390 27615 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.147126 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:16.151109 27565 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:18:16.140161 26365 debug-util.cc:398] Leaking SignalData structure 0x7b0800320d40 after lost signal to thread 25812
I20260709 12:18:16.154987 27752 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.155602 27565 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:16.140164 27623 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.162154 27752 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:16.162972 27850 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.163561 27850 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:16.163817 27623 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.164027 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:16.162992 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:16.165697 27794 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:16.166304 27794 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:16.167380 27794 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.170774 27829 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.179417 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:16.180208 25705 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:16.180615 27829 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.183769 27641 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:16.184183 27641 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:16.185158 27641 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.186125 27641 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:16.182283 27773 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.189162 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.189538 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:16.191447 27793 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.004s
I20260709 12:18:16.191603 27774 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.192780 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.194043 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.196004 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.197360 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.197965 25707 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:16.176316 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1414) took 2486 ms. Trace:
I20260709 12:18:16.198884 25781 rpcz_store.cc:276] 0709 12:18:13.690328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.644088 (+1953760us) service_pool.cc:224] Handling call
0709 12:18:15.673189 (+ 29101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.864205 (+191016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.176005 (+311800us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400098654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:16.176225 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433443584}
I20260709 12:18:16.176573 25787 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:16.177119 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1981) took 1313 ms. Trace:
I20260709 12:18:16.201174 25711 rpcz_store.cc:276] 0709 12:18:14.863737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.193838 (+330101us) service_pool.cc:224] Handling call
0709 12:18:15.259256 (+ 65418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.390062 (+130806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.176867 (+786805us) spinlock_profiling.cc:243] Waited 89.8 ms on lock 0x7b5400073fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:16.177035 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206541056}
I20260709 12:18:16.160363 27855 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:18:16.216825 25705 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:16.218559 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.220041 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:16.220281 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.220896 25711 raft_consensus.cc:2368] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5.
I20260709 12:18:16.221796 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.222820 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:16.220899 25707 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:16.139746 27831 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.224017 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.230849 27554 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.232048 27821 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.178373 27823 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:16.235268 27624 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.001s
I20260709 12:18:16.246097 27624 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.261668 27624 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.163990 27850 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:16.264185 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.264631 27831 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.265556 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1916) took 1222 ms. Trace:
I20260709 12:18:16.265946 25710 rpcz_store.cc:276] 0709 12:18:15.043506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.264685 (+1221179us) service_pool.cc:224] Handling call
0709 12:18:16.264995 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.265224 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.265251 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.265297 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:16.265326 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.265377 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.265470 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.267277 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:16.267975 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.277045 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1968) took 1422 ms. Trace:
W20260709 12:18:16.261688 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.256105 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.191915 27793 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:16.242937 26365 debug-util.cc:398] Leaking SignalData structure 0x7b0800331c00 after lost signal to thread 26368
W20260709 12:18:16.279258 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.355310 27781 consensus_queue.cc:1175] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0ae979356d694a09be4b0cd976ea354a" responder_term: 4 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: [3.5-3.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:16.285094 27824 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.285862 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.286459 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.224471 27568 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.287220 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.287711 25783 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:16.287976 27824 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.289388 27568 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:16.252892 27519 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.267902 25710 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:16.254663 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.255059 27821 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.255505 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1911) took 1230 ms. Trace:
I20260709 12:18:16.256772 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:16.139603 27852 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 12:18:16.257642 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1912) took 1232 ms. Trace:
I20260709 12:18:16.295218 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.258029 25784 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:16.258386 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.258762 27825 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.259009 27748 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.272086 27554 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.222394 25711 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:16.270536 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1987) took 1243 ms. Trace:
I20260709 12:18:16.274845 25709 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
W20260709 12:18:16.306248 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 68173.2us
I20260709 12:18:16.306470 25705 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:16.276216 27615 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.246109 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.277376 27534 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.319890 27534 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.320405 27534 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.283545 25637 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:16.323027 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.324213 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.287859 25782 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:16.277436 25706 rpcz_store.cc:276] 0709 12:18:14.855118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.855757 (+   639us) service_pool.cc:224] Handling call
0709 12:18:14.856291 (+   534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.538948 (+682657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.276764 (+737816us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54003abe54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:16.276957 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562048896}
I20260709 12:18:16.295594 25708 rpcz_store.cc:276] 0709 12:18:15.025269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.240695 (+1215426us) service_pool.cc:224] Handling call
0709 12:18:16.257122 (+ 16427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.257332 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.257360 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.257403 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:16.257432 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.257484 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.257589 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.304308 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.294740 25713 rpcz_store.cc:276] 0709 12:18:15.024845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.236756 (+1211911us) service_pool.cc:224] Handling call
0709 12:18:16.237137 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.239256 (+  2119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.255151 (+ 15895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.255206 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:16.255248 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.255315 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.255445 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.308569 27850 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.309553 27748 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.311154 25635 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:16.313354 25631 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:16.334229 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.313941 25711 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:16.315034 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.335320 25781 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:16.336477 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.315673 26339 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
W20260709 12:18:16.337668 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.315865 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.316210 25786 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:16.316908 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1921) took 1271 ms. Trace:
I20260709 12:18:16.317162 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1887) took 1033 ms. Trace:
I20260709 12:18:16.342355 25787 rpcz_store.cc:276] 0709 12:18:15.284167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.802952 (+518785us) service_pool.cc:224] Handling call
0709 12:18:16.176196 (+373244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.200803 (+ 24607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.316961 (+116158us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400098650. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:16.317099 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825656704}
I20260709 12:18:16.287787 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.318145 27825 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:16.318758 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.348016 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.348444 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.319504 25633 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:16.327108 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1919) took 1282 ms. Trace:
W20260709 12:18:16.346602 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 197721us
I20260709 12:18:16.349362 25705 rpcz_store.cc:276] 0709 12:18:15.044552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.277375 (+1232823us) service_pool.cc:224] Handling call
0709 12:18:16.306307 (+ 28932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.306980 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.326929 (+ 19949us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b54002238d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:16.327049 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65733504}
W20260709 12:18:16.329108 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.343724 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.345002 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1920) took 1300 ms. Trace:
I20260709 12:18:16.341182 25709 rpcz_store.cc:276] 0709 12:18:15.045354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.306519 (+1261165us) service_pool.cc:224] Handling call
0709 12:18:16.306807 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.316447 (+  9640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.316495 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.316554 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:16.316605 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.316682 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.316817 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.350982 25781 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.350997 25711 rpcz_store.cc:276] 0709 12:18:15.044981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.304020 (+1259039us) service_pool.cc:224] Handling call
0709 12:18:16.313712 (+  9692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.344739 (+ 31027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.344944 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.339265 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.353732 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1931) took 1057 ms. Trace:
I20260709 12:18:16.354249 25711 rpcz_store.cc:276] 0709 12:18:15.296833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.352714 (+1055881us) service_pool.cc:224] Handling call
0709 12:18:16.353091 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.353313 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.353350 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.353405 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:16.353481 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.353556 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.353671 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.353852 27762 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:16.356297 27762 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.356812 27762 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:16.359724 25781 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:16.361584 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1883) took 1078 ms. Trace:
I20260709 12:18:16.362147 25786 rpcz_store.cc:276] 0709 12:18:15.282829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.685321 (+402492us) service_pool.cc:224] Handling call
0709 12:18:15.685659 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.339239 (+653580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.347932 (+  8693us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54001257d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:16.361508 (+ 13576us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449603328}
I20260709 12:18:16.348636 25785 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:16.350049 25603 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:16.361650 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 105004us
I20260709 12:18:16.365401 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:16.367292 25705 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:16.368046 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1926) took 1244 ms. Trace:
I20260709 12:18:16.368353 25705 rpcz_store.cc:276] 0709 12:18:15.123433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.350401 (+1226968us) service_pool.cc:224] Handling call
0709 12:18:16.367128 (+ 16727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.367807 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.367992 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.369674 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:16.370476 25706 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:16.370596 27766 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.371374 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1988) took 1307 ms. Trace:
I20260709 12:18:16.371752 25708 rpcz_store.cc:276] 0709 12:18:15.063547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.345678 (+1282131us) service_pool.cc:224] Handling call
0709 12:18:16.370895 (+ 25217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.371060 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.371090 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.371130 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:16.371157 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.371204 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.371291 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.372730 27766 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.373656 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.361303 25711 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:16.377058 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:16.379528 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.380461 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.381883 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.315886 27519 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.382746 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:16.333372 27793 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.383383 26335 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:16.383817 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:16.385591 27828 consensus_meta.cc:287] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.505s	user 0.014s	sys 0.004s
I20260709 12:18:16.387876 27795 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.307875 25704 rpcz_store.cc:276] 0709 12:18:15.027365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.241677 (+1214312us) service_pool.cc:224] Handling call
0709 12:18:16.262937 (+ 21260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.269628 (+  6691us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:18:16.269670 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.269715 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:16.269759 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.270282 (+   523us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b54000730d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:16.270325 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.270458 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7068,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":14513920}
I20260709 12:18:16.392966 25782 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:16.361896 27579 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:16.393966 25709 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:18:16.394621 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1927) took 1270 ms. Trace:
I20260709 12:18:16.395015 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.394917 25709 rpcz_store.cc:276] 0709 12:18:15.123866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.351369 (+1227503us) service_pool.cc:224] Handling call
0709 12:18:16.390725 (+ 39356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.394362 (+  3637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.394570 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.391073 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:16.398147 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:16.399353 25782 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:16.392752 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.406388 25786 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:16.407932 25677 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.386013 27828 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.348294 27825 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:16.411764 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.413656 25705 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:16.406225 27624 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:16.416352 27624 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:16.416756 27624 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:16.418831 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:16.374341 27554 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.423938 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
W20260709 12:18:16.440478 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 297854us
I20260709 12:18:16.440788 25785 raft_consensus.cc:606] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:16.442132 25706 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:18:16.421976 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:16.429756 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1997) took 1026 ms. Trace:
I20260709 12:18:16.443392 25705 rpcz_store.cc:276] 0709 12:18:15.403282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.406113 (+1002831us) service_pool.cc:224] Handling call
0709 12:18:16.413478 (+  7365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.415788 (+  2310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.429659 (+ 13871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.406401 27821 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.399158 27853 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.452067 25712 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 2: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:16.452406 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 102445us
I20260709 12:18:16.442584 25706 pending_rounds.cc:85] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 2
I20260709 12:18:16.454815 25706 pending_rounds.cc:107] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:18:16.455158 25706 raft_consensus.cc:2889] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:16.458351 25708 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:18:16.406612 27823 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:16.461596 25709 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:16.420672 27825 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:16.472066 25707 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:16.473781 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2001) took 1069 ms. Trace:
W20260709 12:18:16.475009 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 98068.9us
I20260709 12:18:16.453187 27832 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:16.453513 27748 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:16.484665 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.444352 27827 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:16.474972 25603 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:16.414005 27641 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 12:18:16.490610 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 74620.4us
I20260709 12:18:16.410835 27793 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:18:16.492188 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.453083 27821 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:16.493221 27821 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.494669 27651 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:16.495110 27651 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:16.495464 27651 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:16.487907 25782 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:16.475507 25713 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:16.485469 25779 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:16.489384 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.486655 25706 rpcz_store.cc:276] 0709 12:18:15.404139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.409698 (+1005559us) service_pool.cc:224] Handling call
0709 12:18:16.441821 (+ 32123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.473509 (+ 31688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.473710 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.407091 27747 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.486930 27690 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:18:16.457521 27853 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:16.424650 27579 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:16.489768 25712 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:16.502213 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.502880 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 80977.4us
I20260709 12:18:16.503424 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.506361 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.508431 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.509758 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.510255 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.444572 27675 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:18:16.513758 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:16.516651 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.453033 27831 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:18:16.520327 25784 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:16.487696 27856 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:16.523499 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:16.524061 26343 raft_consensus.cc:2410] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:18:16.525148 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.487593 27748 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.526636 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.527030 25630 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:16.465055 27823 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:16.498627 27453 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:16.523984 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:16.525151 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.532450 25786 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:16.456001 27554 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.521355 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:16.531142 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:16.533910 25779 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.531594 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:16.479825 27661 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:18:16.527053 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:16.528589 26339 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 3: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:16.528920 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.501313 27832 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:16.526451 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:16.519610 27853 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.475993 27488 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:18:16.491472 27855 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:18:16.526595 27623 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.523308 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 1998) took 1116 ms. Trace:
I20260709 12:18:16.525738 27827 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:16.506290 27773 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:18:16.535298 27823 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:16.531563 27748 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:16.560057 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1914) took 1517 ms. Trace:
I20260709 12:18:16.535543 27675 consensus_queue.cc:1243] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:16.534858 27453 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:16.566922 25707 rpcz_store.cc:276] 0709 12:18:15.042812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.242407 (+1199595us) service_pool.cc:224] Handling call
0709 12:18:16.468125 (+225718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.477265 (+  9140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.559855 (+ 82590us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400104850. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:16.559995 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484192640}
I20260709 12:18:16.568375 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:16.568547 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:16.576498 25629 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.576640 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:16.524358 27849 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:18:16.559756 27623 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.580533 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:16.580247 25635 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:16.602643 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.536767 27767 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:18:16.551294 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:16.535739 27851 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:16.559194 27795 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:16.579928 27453 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:16.642055 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 167185us
I20260709 12:18:16.578786 27754 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:16.660871 25786 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:16.558796 27690 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:18:16.575493 27856 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.667527 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.645756 27623 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.670102 25603 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:16.675295 25632 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.680822 25629 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:16.568285 27863 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 12:18:16.690893 25632 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:16.702860 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:16.704690 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 136024us
I20260709 12:18:16.706709 25603 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:16.708264 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.709599 25603 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:16.559780 25704 rpcz_store.cc:276] 0709 12:18:15.403459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.406918 (+1003459us) service_pool.cc:224] Handling call
0709 12:18:16.423604 (+ 16686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.460405 (+ 36801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.466367 (+  5962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.484383 (+ 18016us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:16.486893 (+  2510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.505800 (+ 18907us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.519398 (+ 13598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.724611 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:16.705871 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:16.683012 27875 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.733466 27872 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.733978 27872 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:16.734375 27872 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.721575 27690 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:16.736907 27747 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.705789 27855 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.738992 27855 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.739375 27855 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:16.739955 27690 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:16.740988 27690 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.743409 27747 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.743860 27747 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:16.744220 27747 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:16.744306 27864 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:18:16.718195 27773 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:16.744379 27854 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:16.773432 25779 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:16.719938 27754 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:16.777076 25787 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
W20260709 12:18:16.774729 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 70286.6us
I20260709 12:18:16.663436 27579 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:16.727746 27828 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.673250 27832 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.772323 27858 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:16.790784 27870 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:16.794845 27850 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:16.698052 27856 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.776010 27773 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:16.779497 27854 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:16.782387 27754 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:16.832531 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.863869 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.808415 27554 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:16.967247 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.727447 27827 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:16.816821 27828 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.808444 27680 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.809926 27870 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:17.023826 25711 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.023s	sys 0.035s
I20260709 12:18:16.819326 27858 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:16.766729 27875 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.028172 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:17.028033 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:16.827325 27773 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:16.886168 27615 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:18:17.037552 25711 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.023s	sys 0.035s
I20260709 12:18:17.024282 27568 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.811498 27872 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:17.099536 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.912681 27754 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:17.142098 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 114027us
I20260709 12:18:17.085237 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:17.004212 27851 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
W20260709 12:18:17.023203 27774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.017s	sys 0.041s
I20260709 12:18:17.161418 25778 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:17.142729 27615 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:17.156656 27762 consensus_meta.cc:287] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.799s	user 0.007s	sys 0.010s
I20260709 12:18:17.036338 27554 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.131s
W20260709 12:18:17.180398 27821 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.672s	user 0.008s	sys 0.003s
I20260709 12:18:16.887107 27624 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:17.151448 27847 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:18:17.082046 27680 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:17.182541 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 476723us
I20260709 12:18:17.220885 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:17.036295 27828 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.151005 27881 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.164891 27883 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:18:17.181262 27774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.037s	sys 0.047s
I20260709 12:18:17.157388 27568 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:17.209066 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.204823 27821 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.150296 27855 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.202790 27762 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.252480 27856 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.222772 27870 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.197532 27854 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.252182 27888 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.252702 27883 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:17.363477 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 265) took 1284 ms. Trace:
I20260709 12:18:17.376374 25711 rpcz_store.cc:276] 0709 12:18:16.079011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.468707 (+389696us) service_pool.cc:224] Handling call
0709 12:18:16.469085 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.469356 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:17.140048 (+670692us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.362994 (+222946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.363066 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:17.363128 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.363232 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.363401 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":108318,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"thread_start_us":807451,"threads_started":2}
W20260709 12:18:17.339095 27878 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 12:18:17.412832 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.004s	sys 0.010s
W20260709 12:18:17.415046 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.004s	sys 0.010s
W20260709 12:18:17.345378 27879 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:18:17.201655 27889 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.118s
W20260709 12:18:17.416297 27823 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.835s	user 0.006s	sys 0.003s
W20260709 12:18:17.416755 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 637055us
I20260709 12:18:17.416788 27823 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.429229 27823 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:17.326133 27748 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.008s
W20260709 12:18:17.334044 27453 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.011s
I20260709 12:18:17.322383 27772 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:18:17.432762 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.333526 27624 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.371024 27831 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:17.338316 27872 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:17.375123 25704 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:17.304509 27615 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.346834 27858 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.345470 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:17.340314 27821 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:17.465965 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 167515us
I20260709 12:18:17.461164 27680 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.358403 27892 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:17.476183 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.464402 27825 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.975s	user 0.010s	sys 0.006s
I20260709 12:18:17.463694 27748 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:17.488600 27825 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.490882 27825 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.491257 27872 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:17.493930 27748 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.478950 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:17.494630 27748 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:17.494737 25787 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:17.495039 27748 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:17.495888 27748 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.479311 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.500517 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:17.501605 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:17.503243 27886 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:18:17.504199 27883 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.507104 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.507833 25783 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:17.508891 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.510057 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.510898 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:17.511706 25779 raft_consensus.cc:2368] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3.
I20260709 12:18:17.512574 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.507977 27754 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.479861 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:17.480448 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.480654 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.516078 25781 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
W20260709 12:18:17.467949 27651 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.972s	user 0.013s	sys 0.000s
I20260709 12:18:17.235118 27881 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.483978 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:17.461545 27772 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:17.525159 27772 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.485677 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.373754 27880 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 12:18:17.526384 25782 raft_consensus.cc:2376] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3. Re-sending same reply.
I20260709 12:18:17.481204 27719 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.487247 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.481685 27870 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:17.481925 27793 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:17.527456 25785 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:17.518278 25779 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:17.515949 25784 raft_consensus.cc:2376] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5. Re-sending same reply.
I20260709 12:18:17.521039 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.517381 25787 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:17.530920 27719 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.531289 27850 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:17.532004 27898 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:17.532292 27895 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:17.532912 27897 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:17.519320 27651 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:17.536876 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.560398 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.537710 27534 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.217s	user 0.007s	sys 0.007s
I20260709 12:18:17.451459 27888 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.540246 27881 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.541251 27875 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:17.541747 27568 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:17.542174 27855 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.541057 27821 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:18:17.543778 27828 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.545641 27829 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:17.547873 27892 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:17.464250 27453 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.553582 27825 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:17.561321 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:17.571138 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1924) took 2524 ms. Trace:
I20260709 12:18:17.718020 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:17.721062 25629 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:17.721146 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:17.721750 25635 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:17.555495 27747 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.004s
I20260709 12:18:17.704079 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:17.726342 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:17.727088 25632 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:17.698932 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 278996us
I20260709 12:18:17.730397 25786 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:17.726440 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:17.712779 25637 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:17.678061 27901 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:17.722685 25630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:37014
I20260709 12:18:17.722385 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:17.726120 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:17.678643 27534 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.710798 27880 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:18:17.718182 25710 rpcz_store.cc:276] 0709 12:18:15.046647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.330975 (+1284328us) service_pool.cc:224] Handling call
0709 12:18:16.362284 (+ 31309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.374690 (+ 12406us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:16.516803 (+142113us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.526065 (+1009262us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b180038f980. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a 00007f29b5999a2a
0709 12:18:17.570693 (+ 44628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.570762 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:17.570818 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.570905 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.571066 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142876,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":12480,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":14699776,"thread_start_us":117263,"threads_started":2}
I20260709 12:18:17.741219 25707 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:17.741535 27875 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:17.741981 25706 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:17.741901 27675 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:17.742389 27554 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.769670 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:17.720237 27623 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.021s	user 0.012s	sys 0.000s
W20260709 12:18:17.746661 25778 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.003s
I20260709 12:18:17.747243 27747 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.747886 27719 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:17.747769 27825 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:17.754434 27888 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:17.660413 27793 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:17.710668 27829 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:17.766095 25780 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:17.768428 25638 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:17.761821 25630 raft_consensus.cc:606] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:17.769210 25633 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:17.850550 25635 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:17.742702 27661 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:17.853582 25708 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:17.745168 27453 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.767526 27824 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:17.834975 27852 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:17.761941 27519 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:17.836261 27754 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:17.767226 27766 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:17.841955 27875 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:17.863147 27881 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:17.865578 27773 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.702s	user 0.012s	sys 0.000s
I20260709 12:18:17.861229 27880 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:17.765776 27651 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.898641 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:17.873248 25632 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:17.758436 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:17.761545 27821 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.122s
I20260709 12:18:17.901834 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:17.908231 27824 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:17.902285 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:17.864871 25778 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:17.905695 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:17.981262 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 75658.5us
I20260709 12:18:17.759158 25704 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:17.909440 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:17.910342 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 354473us
I20260709 12:18:17.982580 25638 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:17.983454 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:17.983947 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:17.769158 27902 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:17.765816 27534 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:17.726949 27898 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:17.759618 27762 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:17.863029 27623 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.991609 27623 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:17.863503 27554 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.993943 27748 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:17.994542 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:17.996003 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:17.996762 25632 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:17.994040 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:18.003064 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:18.004508 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.005985 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:17.915457 27690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.023s	sys 0.033s
W20260709 12:18:18.010327 27690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.023s	sys 0.035s
I20260709 12:18:18.011946 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.012735 25632 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:18.013495 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.014144 25632 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:18.015003 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:17.761222 27901 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:18.016088 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.017163 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.017889 25633 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:18.017731 27901 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:18.019867 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.024441 27828 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:18.025913 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:18.026273 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:17.986934 27829 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.892750 27892 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:17.947822 25710 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:17.948516 25636 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:17.903055 25637 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:17.949796 27906 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:18.035212 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:17.953279 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:18:17.943073 27773 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:17.936631 27579 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:18:17.955129 27793 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:17.936793 27766 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:18.033895 27858 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:17.973666 27747 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:17.915201 27852 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:17.933611 27519 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:17.988804 27898 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:18.069018 27912 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:18:18.008121 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:17.975878 27880 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:17.984252 27853 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.418s	user 0.013s	sys 0.000s
I20260709 12:18:18.009549 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.083446 25631 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:17.991988 27824 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.028239 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.084438 25634 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:18.030160 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:18.031081 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.034582 26343 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 4 LEARNER]: Advancing to term 5
I20260709 12:18:18.027887 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.038034 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:18.031601 25708 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:17.946439 25779 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:17.951031 27875 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:18.056797 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:18.031297 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 318) took 1565 ms (client timeout 1934 ms). Trace:
I20260709 12:18:17.972867 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 612) took 1473 ms. Trace:
I20260709 12:18:17.985562 27902 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:18.007189 27825 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:18.080435 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:18.106451 25710 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.055994 27821 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:18.083748 25707 raft_consensus.cc:1813] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Handling vote request from an unknown peer 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:18.107331 25707 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:18.087118 25603 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:18.089203 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:18.089412 26338 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:18.089780 27828 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:18.078382 26335 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:18.031312 27907 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.095208 25633 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:18.095764 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.071452 27906 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.085263 27858 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:18.066291 27892 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.109488 25634 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:18.099972 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 64850.2us
I20260709 12:18:18.056207 27651 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:18:18.092619 27519 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:18.197605 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.337373 25711 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:18.115717 25782 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:18:18.205797 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.340615 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.341068 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:18.341763 25708 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:18.339282 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:18.342157 25636 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:18.343262 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.343446 26341 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:18.158242 25710 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:18.124955 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 316) took 1768 ms. Trace:
I20260709 12:18:18.244774 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:18.192875 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:18.163282 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.095012 27853 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.105458 27901 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.290s
I20260709 12:18:18.108495 27579 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:18.419245 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:18.372712 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:18.426724 27853 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:18.427053 26338 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:18.411615 27719 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.363588 26343 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 5 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:18.427865 27719 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.380751 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:18.428203 27858 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:18.429538 25533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 302) took 6705 ms. Trace:
I20260709 12:18:18.430081 27719 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:18.440160 27889 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:18.103170 25778 rpcz_store.cc:269] 0709 12:18:16.420396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.013452 (+593056us) service_pool.cc:224] Handling call
0709 12:18:17.985331 (+971879us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:18.358615 27902 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.399346 27772 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.441951 27902 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:18.443840 27901 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:18.444245 27901 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:18.445019 27901 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:18.369923 25629 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:18.446099 27901 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.354260 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.201550 27875 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:18.372221 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:18.355580 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.194875 27880 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:18.352831 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.449998 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:18.354981 27827 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.376984 27854 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.209743 27766 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.388526 27829 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.397677 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:18.453438 25633 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.351480 27554 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:18.263772 27850 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:18.402661 27892 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:18.393193 27906 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:18.405165 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 66029.5us
W20260709 12:18:18.112668 27831 consensus_queue.cc:1175] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "955233c139344131a847044f17f8d348" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:18.460621 27766 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.247754 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:18.347085 25707 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.463727 25632 raft_consensus.cc:2368] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3.
W20260709 12:18:18.408365 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 380500us
I20260709 12:18:18.346796 25780 rpcz_store.cc:276] 0709 12:18:16.356172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.383543 (+ 27371us) service_pool.cc:224] Handling call
0709 12:18:18.124726 (+1741183us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540009a454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:18.124884 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3177280896}
I20260709 12:18:18.465591 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:18.409587 25631 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:18.395016 27915 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.184s	user 0.000s	sys 0.003s
W20260709 12:18:18.348191 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 623488us
W20260709 12:18:18.263949 27893 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.799s	user 0.005s	sys 0.008s
I20260709 12:18:18.113615 27747 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:18.377458 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:18.420213 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:18.423666 25634 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:18.348709 27795 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:18.350310 27794 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:18:18.416466 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:18.424598 25534 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:18.425360 27579 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:18.414587 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 251452us
I20260709 12:18:18.432165 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 595303b3057546cbbfda4b84f8df8912 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51510
I20260709 12:18:18.414284 27876 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:18.413371 27907 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:18.434922 25705 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:18.361102 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:18.107664 27785 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.328s
I20260709 12:18:18.435994 27767 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:18.131170 27821 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:18.118554 27852 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.437448 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:18.339763 27898 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:18.447008 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:18.447304 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 441863us
I20260709 12:18:18.231845 27615 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.344990 27824 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.448673 25713 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:18.450003 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.555262 25712 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:18.555361 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.450873 25637 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:18.555820 26309 connection.cc:256] RPC call timeout handler was delayed by -1.0081496989998868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
I20260709 12:18:18.556133 25711 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 3: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:18.556836 25637 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:18:18.556790 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:18.450856 27888 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:18.450995 27880 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:18.546947 27795 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:18.447103 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:18.574126 25707 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:18.467788 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:18.450834 27827 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.467164 27832 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.574120 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:18.587430 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 140352us
I20260709 12:18:18.588037 25637 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:18.504200 25635 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:18.546655 25710 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:18.471501 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.484445 25632 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:18.548151 27568 consensus_queue.cc:1175] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "10d0c7349085463595aedaf7c071e742" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 12:18:18.589579 25630 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:18.470183 25631 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:18.473873 27872 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:18.566501 27919 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:18:18.567301 27781 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:18.567710 27690 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:18.549448 25629 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:18.590879 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 123159us
I20260709 12:18:18.590971 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:18.585296 25534 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:18.591710 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 44127.7us
I20260709 12:18:18.546092 27767 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:18.585976 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:18.587715 25711 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:18.509601 25638 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:18.490442 27773 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:18.549973 27579 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:18.554704 27747 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:18.575372 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.593050 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.615439 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 266) took 2506 ms. Trace:
I20260709 12:18:18.585844 25634 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:18.615852 25705 rpcz_store.cc:276] 0709 12:18:16.108943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.478249 (+369306us) service_pool.cc:224] Handling call
0709 12:18:16.486692 (+  8443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.550253 (+2063561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.615192 (+ 64939us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b5400146754. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:18.615379 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4480045952}
I20260709 12:18:18.551076 27876 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:18.595760 25708 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:18.596390 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.591666 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:18.599452 25631 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:18.596458 25534 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:18.601183 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.571189 27898 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.602406 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.600699 27554 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:18.600864 27680 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:18.606739 27794 consensus_queue.cc:1243] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:18.105468 26337 rpcz_store.cc:276] 0709 12:18:16.499875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.505713 (+  5838us) service_pool.cc:224] Handling call
0709 12:18:17.523538 (+1017825us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aeee 000000000036eb39 00007f29ba8de1e0 00007f29ba9a70c4 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:17.523715 (+   177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.524243 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.524271 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.524819 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:18:17.864182 (+339363us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:17.869712 (+  5530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.893070 (+ 23358us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:17.972477 (+ 79407us) raft_consensus.cc:1685] finished
0709 12:18:17.972509 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.972784 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2506,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":4471,"spinlock_wait_cycles":30992128,"thread_start_us":328351,"threads_started":1,"wal-append.queue_time_us":417649}
I20260709 12:18:18.430244 25533 rpcz_store.cc:276] 0709 12:18:11.724178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.724360 (+   182us) service_pool.cc:224] Handling call
0709 12:18:18.429384 (+6705024us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:16.237170 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:16.240297 (+  3127us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:16.240372 (+    75us) write_op.cc:435] Acquired schema lock
0709 12:18:16.240413 (+    41us) tablet.cc:662] Decoding operations
0709 12:18:16.309306 (+ 68893us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:16.309389 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:16.309419 (+    30us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:18:16.310661 (+  1242us) tablet.cc:701] Row locks acquired
0709 12:18:16.310714 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:18:16.310900 (+   186us) write_op.cc:270] Start()
0709 12:18:16.311064 (+   164us) write_op.cc:276] Timestamp: P: 1783599496310833 usec, L: 0
0709 12:18:16.311086 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:18:16.311754 (+   668us) log.cc:844] Serialized 5566 byte log entry
0709 12:18:16.538768 (+227014us) op_driver.cc:464] REPLICATION: finished
0709 12:18:16.825131 (+286363us) write_op.cc:301] APPLY: starting
0709 12:18:16.825271 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:16.825603 (+   332us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:16.825744 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:17.043489 (+217745us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:17.044012 (+   523us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:18:17.044081 (+    69us) write_op.cc:312] APPLY: finished
0709 12:18:17.044592 (+   511us) log.cc:844] Serialized 101 byte log entry
0709 12:18:17.044847 (+   255us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:17.045063 (+   216us) write_op.cc:454] Released schema lock
0709 12:18:17.045160 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:17.045626 (+   466us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:17.045651 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":246381,"apply.run_cpu_time_us":6490,"apply.run_wall_time_us":227984,"num_ops":14,"prepare.queue_time_us":69295,"prepare.run_cpu_time_us":25634,"prepare.run_wall_time_us":117362,"raft.queue_time_us":149162,"raft.run_cpu_time_us":27848,"raft.run_wall_time_us":158433,"replication_time_us":223094,"thread_start_us":267355,"threads_started":4,"wal-append.queue_time_us":42649,"wal-append.run_cpu_time_us":16417,"wal-append.run_wall_time_us":1693310}]]}
I20260709 12:18:18.619282 25711 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:18:18.619642 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.621204 25633 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:18.667680 25705 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:18:18.668376 25629 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:18.595067 27892 raft_consensus.cc:2764] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:18:18.672089 25638 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:18.651963 27917 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:18:18.651841 27767 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:18.641350 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.652817 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:18.653213 25601 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.653007 27795 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.601711 27690 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:18.655567 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.651130 25534 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 2 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:18.685585 25534 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 4 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:18.685971 27898 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.687350 25784 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:18:18.687125 25534 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 3, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:18.659847 25710 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:18.594339 27772 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:18.668603 25705 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:18.699450 25705 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:18.662112 27902 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:18.597429 27781 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.606264 27882 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:18:18.667230 25751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.576s (SENT) [suppressed 31 similar messages]
I20260709 12:18:18.622453 27889 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:18.692652 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:18.726130 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 58951.7us
I20260709 12:18:18.622670 27876 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:18.732725 25706 raft_consensus.cc:2410] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:18.688834 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:18.722398 27623 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:18.734982 25787 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:18.738961 27747 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:18.738981 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:18.671217 25711 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:18:18.749998 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:18.594007 27872 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:18.671082 27554 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:18.712572 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:18.712764 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.711208 27902 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:18.703219 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:18.699724 27863 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:18.655692 27680 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:18.722398 25707 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:18.688643 25534 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 3 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:18.767140 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.729000 27889 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:18.729733 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:18.751080 25711 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:18.724502 27754 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.688565 27898 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:18.758870 27915 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:18:18.746810 27795 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:18.760412 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47772.4us
I20260709 12:18:18.754304 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 613) took 1953 ms. Trace:
I20260709 12:18:18.840867 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:18.788092 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.753968 27618 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:18.740603 27876 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:18.830817 25711 pending_rounds.cc:85] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 1
I20260709 12:18:18.859021 25711 pending_rounds.cc:107] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:18:18.859477 25711 raft_consensus.cc:2889] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:18.766037 27902 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:18.855774 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.773079 27767 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.742256 27579 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.772935 27680 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:18.765774 27773 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:18.841563 27880 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:18.771571 27889 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:18.685602 27821 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:18.869611 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:18.906939 25780 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:18.892282 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 104343us
W20260709 12:18:18.921600 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 256223us
W20260709 12:18:18.866794 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 270480us
I20260709 12:18:18.936717 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:18.870342 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 178761us
I20260709 12:18:18.936841 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.888923 25705 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:18.957803 25712 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:18.963120 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:18.977274 26342 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:18.979354 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:18.982679 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:18.983450 25634 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:18.983788 25634 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:18.984663 25634 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:18.922084 27907 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.005857 25634 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:19.002918 25705 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:18.921727 27827 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:18.921165 27927 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:18.870688 27870 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:18.840370 27754 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:18.858891 27862 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.148s
I20260709 12:18:19.005110 27882 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:18.957074 27880 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.018390 25632 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:18.856972 27748 raft_consensus.cc:2764] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:18:19.023597 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:57968 (request call id 2005) took 1255 ms. Trace:
I20260709 12:18:19.037708 25705 rpcz_store.cc:276] 0709 12:18:17.755184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.621402 (+866218us) service_pool.cc:224] Handling call
0709 12:18:19.010579 (+389177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.015810 27907 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:19.008525 27773 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.022471 27898 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:19.024506 27747 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:19.017580 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54602.2us
W20260709 12:18:19.019898 27901 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
I20260709 12:18:19.014700 27618 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:19.020918 27832 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:19.012383 27772 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:19.031054 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 439440us
I20260709 12:18:19.059931 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:19.058068 25634 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:18:18.923841 27554 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.080775 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:19.060899 27928 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:19.074784 27827 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:19.073473 27690 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:19.083169 27781 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.060395 27773 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.070006 27923 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:18:19.062530 27747 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.066110 27937 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:18:19.069723 27927 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:18:19.067806 27931 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:18:19.121127 27519 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:19.087074 27901 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:18:19.082751 27888 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:18:19.123884 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2011) took 1357 ms. Trace:
I20260709 12:18:19.128504 25706 rpcz_store.cc:276] 0709 12:18:17.756813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.007828 (+251015us) service_pool.cc:224] Handling call
0709 12:18:19.108918 (+1101090us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54003ab454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:19.114422 (+  5504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495900032}
I20260709 12:18:19.123150 27754 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.099442 27876 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:19.092193 27850 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
W20260709 12:18:19.131297 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.131809 27926 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:19.093317 27928 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.133802 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2022) took 1296 ms. Trace:
W20260709 12:18:19.133885 27925 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:18:19.125566 27579 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.139814 25706 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:19.174820 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1950) took 1137 ms. Trace:
I20260709 12:18:19.140885 27762 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:18:19.145571 27829 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.690s	user 0.007s	sys 0.000s
W20260709 12:18:19.146181 27615 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.575s	user 0.003s	sys 0.004s
I20260709 12:18:19.147686 27773 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:18.845995 26340 rpcz_store.cc:276] 0709 12:18:16.800962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.803828 (+  2866us) service_pool.cc:224] Handling call
0709 12:18:16.804313 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.804826 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.804873 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.805556 (+   683us) log.cc:844] Serialized 26 byte log entry
0709 12:18:17.199236 (+393680us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:17.199710 (+   474us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.222847 (+ 23137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.228861 (+  6014us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.263332 (+1034471us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:18.672635 (+409303us) raft_consensus.cc:1685] finished
0709 12:18:18.711432 (+ 38797us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.747397 (+ 35965us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1101,"prepare.queue_time_us":183,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":20747,"raft.run_cpu_time_us":518,"raft.run_wall_time_us":226117,"spinlock_wait_cycles":78929280,"thread_start_us":375851,"threads_started":1,"wal-append.queue_time_us":493209}
I20260709 12:18:19.151849 27880 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:19.152719 27929 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:18:19.135155 27854 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
W20260709 12:18:19.154832 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58017.5us
W20260709 12:18:19.155366 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.197575 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 186339us
W20260709 12:18:19.198318 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.122809 27935 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:18:19.168512 25711 rpcz_store.cc:276] 0709 12:18:17.837726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.830037 (+992311us) service_pool.cc:224] Handling call
0709 12:18:18.830571 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.879047 (+ 48476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.879097 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.879720 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.016232 (+136512us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:19.016269 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.017386 (+  1117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.133584 (+116198us) raft_consensus.cc:1685] finished
0709 12:18:19.133608 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.133743 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":95871,"threads_started":1,"wal-append.queue_time_us":152535}
I20260709 12:18:19.098671 27932 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:18:19.175185 25706 rpcz_store.cc:276] 0709 12:18:18.037003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.129560 (+1092557us) service_pool.cc:224] Handling call
0709 12:18:19.139639 (+ 10079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.174543 (+ 34904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.174757 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.179052 25637 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:18:19.185774 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1775) took 1198 ms. Trace:
W20260709 12:18:19.191254 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 103680us
I20260709 12:18:19.165238 27936 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:19.172423 27944 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.198200 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:19.203037 25630 rpcz_store.cc:276] 0709 12:18:17.981548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.981719 (+   171us) service_pool.cc:224] Handling call
0709 12:18:19.178383 (+1196664us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54003007d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:19.179417 (+  1034us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158813568}
W20260709 12:18:19.209712 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 340215us
I20260709 12:18:19.203622 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:19.188020 27766 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 12:18:19.210505 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:19.211298 26339 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:19.212924 27773 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:19.213248 25706 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:18:19.212724 27944 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.213876 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1956) took 1141 ms. Trace:
I20260709 12:18:19.214263 25706 rpcz_store.cc:276] 0709 12:18:18.073120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.212677 (+1139557us) service_pool.cc:224] Handling call
0709 12:18:19.213037 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.213622 (+   585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.213825 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.215879 27944 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.219331 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:19.220315 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 767352us
I20260709 12:18:19.220841 27934 raft_consensus.cc:2764] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:18:19.221663 27934 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:19.223372 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1800) took 1101 ms. Trace:
I20260709 12:18:19.223759 25632 rpcz_store.cc:276] 0709 12:18:18.121942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.589668 (+467726us) service_pool.cc:224] Handling call
0709 12:18:18.615471 (+ 25803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.037272 (+421801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.223149 (+185877us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b5400364a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:19.223315 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":926105472}
W20260709 12:18:19.232601 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.233258 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.211719 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:19.245399 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.282296 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 59288.9us
I20260709 12:18:19.194710 25704 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:19.285796 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1794) took 1165 ms. Trace:
I20260709 12:18:19.287956 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:19.289301 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:19.295775 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.322582 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:19.228132 27880 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:19.209504 27936 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:19.323302 26309 connection.cc:256] RPC call timeout handler was delayed by -1.19137628999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 12:18:19.461550 26309 connection.cc:256] RPC call timeout handler was delayed by -1.190993208000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 12:18:19.386369 27863 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.614s	user 0.008s	sys 0.000s
I20260709 12:18:19.246440 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:19.287873 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:19.466971 25630 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:19.469342 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:19.347723 27939 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:18:19.484747 25629 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:19.485901 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:19.486792 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:19.489396 27898 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:19.493194 27902 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.004s
I20260709 12:18:19.493410 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2016) took 1735 ms. Trace:
I20260709 12:18:19.493809 27902 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.494473 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 627) took 1112 ms. Trace:
I20260709 12:18:19.461743 27906 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:19.384434 27824 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:19.355973 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2026) took 1339 ms. Trace:
I20260709 12:18:19.387569 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2014) took 1630 ms. Trace:
I20260709 12:18:19.381417 27747 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:19.350694 27888 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:19.388276 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2012) took 1631 ms. Trace:
I20260709 12:18:19.365255 27962 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:19.385435 27785 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:19.526852 27902 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:19.527123 27785 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:19.345422 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 116711us
I20260709 12:18:19.453779 25706 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:19.388146 27927 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:19.528326 27785 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.528786 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:19.528915 27927 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.529423 27785 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:18:19.383029 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.530032 27927 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.530964 26343 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:19.531297 27927 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:19.380515 27680 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.533066 27651 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.533558 27651 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.346135 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:19.535528 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:19.535975 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:19.535975 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:19.377506 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:19.378747 27948 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:18:19.383580 27795 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.002s
I20260709 12:18:19.533516 27829 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:19.539034 27829 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:19.456741 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 245080us
I20260709 12:18:19.458468 26344 raft_consensus.cc:2410] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:18:19.539438 27829 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:19.540555 25634 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:19.540952 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.540663 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:19.541714 27955 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.455154 25636 rpcz_store.cc:276] 0709 12:18:18.120112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.550584 (+430472us) service_pool.cc:224] Handling call
0709 12:18:18.552084 (+  1500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.552472 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.552501 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.588844 (+ 36343us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.938609 (+349765us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:19.052210 (+113601us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.052544 (+   334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.052618 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.285539 (+232921us) raft_consensus.cc:1685] finished
0709 12:18:19.285567 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.285727 (+   160us) inbound_call.cc:177] Queueing success response
0709 12:18:19.375899 (+ 90172us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b48002700d8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29ba8b8aa1 00007f29ba8afb37 00007f29ba8af9ca 00007f29ba9a726f 00007f29ba9a6f37 00007f29ba9bb55c 00007f29ba9bb508 00007f29ba9bb488 00007f29ba9bb443 00007f29ba9ba6ec 00007f29b98e4a96
Metrics: {"prepare.queue_time_us":275135,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":7564,"raft.run_cpu_time_us":4694,"raft.run_wall_time_us":827117,"spinlock_wait_cycles":35559808,"thread_start_us":356804,"threads_started":2,"wal-append.queue_time_us":426642}
I20260709 12:18:19.542207 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:19.128631 27772 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.543037 27747 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:19.542394 27955 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:19.543742 27747 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:19.545213 27955 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.464401 26341 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 3 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:19.545755 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:19.474648 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 228318us
I20260709 12:18:19.475628 27952 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:19.546842 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:19.546926 27952 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.547529 27952 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:19.547873 27906 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:19.548609 27906 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:19.549067 27906 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:19.458716 27880 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.494956 26342 rpcz_store.cc:276] 0709 12:18:18.383375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.383930 (+   555us) service_pool.cc:224] Handling call
0709 12:18:18.384280 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.977959 (+593679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.494236 (+516277us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54005a0fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:19.494408 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363134848}
I20260709 12:18:19.493980 25710 rpcz_store.cc:276] 0709 12:18:17.758175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.691987 (+933812us) service_pool.cc:224] Handling call
0709 12:18:18.692311 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.921946 (+229635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.921981 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.927790 (+  5809us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.166859 (+239069us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:19.166911 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.167014 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.493198 (+326184us) raft_consensus.cc:1685] finished
0709 12:18:19.493220 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.493352 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42793,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":362132,"threads_started":2,"wal-append.queue_time_us":488153}
I20260709 12:18:19.550048 27906 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.552039 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2029) took 1125 ms. Trace:
W20260709 12:18:19.498430 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.552759 27950 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.502508 25785 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:19.409168 27850 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:18:19.553472 27950 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.517982 27534 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:19.554610 27534 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.509658 27831 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:19.555591 27831 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.555974 27831 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:19.556227 27950 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.557199 27950 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:19.499807 25713 rpcz_store.cc:276] 0709 12:18:18.016221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.078910 (+1062689us) service_pool.cc:224] Handling call
0709 12:18:19.085485 (+  6575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.087351 (+  1866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.087380 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.091620 (+  4240us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.118966 (+ 27346us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:19.119024 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.119113 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.355736 (+236623us) raft_consensus.cc:1685] finished
0709 12:18:19.355760 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.355911 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":429,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":8389,"thread_start_us":21403,"threads_started":1,"wal-append.queue_time_us":110681}
I20260709 12:18:19.480429 27863 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:19.483681 27872 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.525722 25708 rpcz_store.cc:276] 0709 12:18:17.757077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.669957 (+912880us) service_pool.cc:224] Handling call
0709 12:18:18.670241 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.029628 (+359387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.029661 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.030233 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.104051 (+ 73818us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:19.104098 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.104159 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.388053 (+283894us) raft_consensus.cc:1685] finished
0709 12:18:19.388078 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.388217 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12007,"raft.queue_time_us":64660,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":386514,"threads_started":2,"wal-append.queue_time_us":239516}
I20260709 12:18:19.560369 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1959) took 1098 ms. Trace:
I20260709 12:18:19.503085 27824 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:19.560796 25708 rpcz_store.cc:276] 0709 12:18:18.461631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.559361 (+1097730us) service_pool.cc:224] Handling call
0709 12:18:19.559883 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.560066 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.560089 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:19.560129 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:19.560164 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.560216 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.560316 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.533929 27680 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.376825 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1491) took 1171 ms. Trace:
I20260709 12:18:19.562937 25637 rpcz_store.cc:276] 0709 12:18:18.205202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.594433 (+389231us) service_pool.cc:224] Handling call
0709 12:18:19.178609 (+584176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.202735 (+ 24126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.376600 (+173865us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54003007d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:19.376753 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327511296}
I20260709 12:18:19.371246 26338 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:19.562304 25708 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:19.564821 25704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2015) took 1806 ms (client timeout 1885 ms). Trace:
I20260709 12:18:19.565171 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1960) took 1103 ms. Trace:
I20260709 12:18:19.565613 25708 rpcz_store.cc:276] 0709 12:18:18.461926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.561868 (+1099942us) service_pool.cc:224] Handling call
0709 12:18:19.562150 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.564956 (+  2806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.565118 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:19.566634 26339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 626) took 1187 ms (client timeout 1514 ms). Trace:
I20260709 12:18:19.567097 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1962) took 1104 ms. Trace:
I20260709 12:18:19.567490 25708 rpcz_store.cc:276] 0709 12:18:18.462437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.566346 (+1103909us) service_pool.cc:224] Handling call
0709 12:18:19.566669 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.566824 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.566846 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:19.566887 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:19.566914 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.566961 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.567049 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.569002 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1963) took 1106 ms. Trace:
I20260709 12:18:19.569594 27831 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.570422 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2030) took 1124 ms. Trace:
I20260709 12:18:19.570837 27829 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.535547 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:19.573598 27827 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:19.576046 27829 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.584010 27690 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:19.584667 27690 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:19.569298 25708 rpcz_store.cc:276] 0709 12:18:18.462718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.568304 (+1105586us) service_pool.cc:224] Handling call
0709 12:18:19.568579 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.568735 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.568757 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:19.568792 (+    35us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:19.568817 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.568864 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.568951 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:19.588120 25709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2010) took 1831 ms (client timeout 1861 ms). Trace:
W20260709 12:18:19.588776 25709 rpcz_store.cc:269] 0709 12:18:17.756551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.014702 (+258151us) service_pool.cc:224] Handling call
0709 12:18:19.587809 (+1573107us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b54001bf654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:19.588028 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2168945536}
I20260709 12:18:19.589820 25708 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:19.593607 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1781) took 1581 ms. Trace:
I20260709 12:18:19.537552 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:19.599422 25630 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:19.475745 27618 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:19.478937 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:19.600829 27876 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:19.601403 27876 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.602355 27876 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.549114 27651 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.606714 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1957) took 1529 ms. Trace:
I20260709 12:18:19.468117 27936 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:19.607453 27719 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:19.608158 27719 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.551378 27880 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.502978 27554 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.570837 25713 rpcz_store.cc:276] 0709 12:18:18.445660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.558395 (+1112735us) service_pool.cc:224] Handling call
0709 12:18:19.569918 (+ 11523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.570106 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.570131 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:19.570170 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:19.570199 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.570283 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.570374 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.631053 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:19.577158 27863 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:19.577623 27831 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.574536 25676 leader_election.cc:400] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3.
I20260709 12:18:19.638154 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:19.638532 25676 leader_election.cc:403] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:18:19.639357 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64958us
I20260709 12:18:19.640098 27863 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:19.640344 25676 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:19.579332 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:19.641309 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 62098.4us
I20260709 12:18:19.641495 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:19.641976 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:19.642597 27554 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.643100 27880 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:19.642918 27829 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3.
I20260709 12:18:19.643877 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:19.644095 27829 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:19.606328 27876 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:19.610540 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.614590 25603 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:19.567107 26339 rpcz_store.cc:269] 0709 12:18:18.379825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.380394 (+   569us) service_pool.cc:224] Handling call
0709 12:18:19.566383 (+1185989us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b5400471654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:19.566557 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799150336}
I20260709 12:18:19.594130 25635 rpcz_store.cc:276] 0709 12:18:18.012371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.418176 (+405805us) service_pool.cc:224] Handling call
0709 12:18:18.503977 (+ 85801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.588529 (+ 84552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.593298 (+1004769us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b54002ac6d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:19.593515 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153559936}
W20260709 12:18:19.618397 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.500167 25712 rpcz_store.cc:276] 0709 12:18:17.757607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.671095 (+913488us) service_pool.cc:224] Handling call
0709 12:18:18.671398 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.005321 (+333923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.387192 (+381871us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54003c91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:19.387475 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658053376}
I20260709 12:18:19.653190 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:19.654119 25634 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:19.552421 25710 rpcz_store.cc:276] 0709 12:18:18.426737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.551128 (+1124391us) service_pool.cc:224] Handling call
0709 12:18:19.551430 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.551602 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.551627 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:19.551667 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:19.551810 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.551878 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.551989 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.607199 25706 rpcz_store.cc:276] 0709 12:18:18.086789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.214990 (+1128201us) service_pool.cc:224] Handling call
0709 12:18:19.215255 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.528641 (+313386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.606421 (+ 77780us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b5400105c54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:19.606654 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99557120}
I20260709 12:18:19.631846 25713 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:19.658172 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:19.636257 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:19.637166 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:19.640478 27863 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:19.639555 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:19.538370 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:19.661458 25632 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:19.665060 25709 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:18:19.644776 27889 consensus_meta.cc:287] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.003s
I20260709 12:18:19.643335 27831 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:18:19.672763 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.675542 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:19.680982 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:19.683408 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:19.684692 25712 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:18:19.688262 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 141057us
I20260709 12:18:19.689162 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:19.669727 27944 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:18:19.679868 27889 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.690711 27961 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:19.691823 27961 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:19.691936 27889 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:19.692329 27961 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.692901 27961 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:19.682291 27962 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.695647 27850 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:19.696106 27850 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.696231 27568 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:19.696635 27927 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.695549 27962 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.697239 27927 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.658280 27955 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:19.585980 27690 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.698777 27955 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:19.699344 27690 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:19.699409 27955 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:19.699728 27927 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:19.696686 27568 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:19.700667 27955 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:19.565368 25704 rpcz_store.cc:269] 0709 12:18:17.757890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.004199 (+246309us) service_pool.cc:224] Handling call
0709 12:18:19.564581 (+1560382us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400236254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:19.564743 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2716331904}
I20260709 12:18:19.701372 27568 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.702668 27568 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:19.697127 27850 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:19.704738 27554 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.704686 27850 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:19.706574 27928 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:19.706808 25637 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:19.635686 27719 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.709451 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2027) took 1303 ms. Trace:
I20260709 12:18:19.711288 27719 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:19.710855 25707 rpcz_store.cc:276] 0709 12:18:18.406307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.224694 (+818387us) service_pool.cc:224] Handling call
0709 12:18:19.225061 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.225459 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.225499 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.226012 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.486669 (+260657us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:19.486743 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.486855 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.598998 (+112143us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b48000cf0d8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29ba8b8aa1 00007f29ba8afb37 00007f29ba8af9ca 00007f29ba9a726f 00007f29ba9a6f37 00007f29ba9bb55c 00007f29ba9bb508 00007f29ba9bb488 00007f29ba9bb443 00007f29ba9ba6ec 00007f29b98e4a96
0709 12:18:19.709154 (+110156us) raft_consensus.cc:1685] finished
0709 12:18:19.709197 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.709395 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2706,"raft.run_cpu_time_us":2463,"raft.run_wall_time_us":357152,"spinlock_wait_cycles":64104832,"thread_start_us":259259,"threads_started":1,"wal-append.queue_time_us":448439}
I20260709 12:18:19.716925 27534 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.719393 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:19.720151 25633 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
W20260709 12:18:19.722119 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.723243 25636 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:18:19.726608 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45725.6us
W20260709 12:18:19.731274 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.738695 26309 leader_election.cc:336] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.702s (SENT)
I20260709 12:18:19.739243 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:19.744870 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:19.748061 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:19.749300 25636 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:19.660072 27773 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:19.759691 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 82669.4us
I20260709 12:18:19.760839 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:19.762418 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:19.718279 27534 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:19.762981 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:19.763381 25784 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:19.724398 27852 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:19.562202 27824 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.781003 27824 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:19.782774 27618 consensus_queue.cc:1175] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0ae979356d694a09be4b0cd976ea354a" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:19.692917 27889 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:18:19.788611 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 43879.4us
I20260709 12:18:19.790690 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:19.793401 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49669.7us
I20260709 12:18:19.765815 27863 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:19.770579 27882 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:19.669430 27934 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:18:19.682668 27831 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:19.763504 27773 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:19.791493 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 52854.3us
I20260709 12:18:19.791998 27651 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:19.796398 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.770129 27568 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:19.801887 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:19.802397 26307 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.029s	sys 0.041s
W20260709 12:18:19.814247 26307 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.029s	sys 0.042s
W20260709 12:18:19.814707 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 531166us
I20260709 12:18:19.760133 27932 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.818229 26341 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:19.819916 26344 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:19.820150 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:19.813194 25708 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:19.820951 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:19.821585 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:19.822250 25706 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:19.819120 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:19.822978 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:19.777953 27936 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:19.840636 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.852452 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 91718.3us
I20260709 12:18:19.853690 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:19.823738 27615 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.812036 27821 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:19.811878 27831 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:19.845021 27850 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:19.827788 26339 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:19.816073 27928 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:19.812077 27773 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:19.806154 27950 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:19.811897 27651 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:19.865561 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.865404 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:19.805963 27882 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:19.874871 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:19.815696 27952 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.824870 27568 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.888019 26337 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:19.851819 27932 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.891482 26336 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:19.816648 27892 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:18:19.864832 27829 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:19.862610 27934 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:18:19.875828 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:19.905483 27923 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.002s	sys 0.014s
I20260709 12:18:19.864221 27615 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.888290 27850 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.850708 27863 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:19.915056 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:19.918849 27902 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.888113 27928 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.907713 27651 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:19.903558 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44345.9us
I20260709 12:18:19.879509 25632 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:19.912752 27882 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:19.875762 27680 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:18:19.907312 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:19.952759 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:19.963583 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:19.963786 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 349319us
I20260709 12:18:20.008196 27579 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:19.945052 27829 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:20.009081 27579 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:20.009594 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 46098us
W20260709 12:18:20.010226 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.009904 27829 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.010309 27579 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.948801 27928 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:20.001433 26309 leader_election.cc:336] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.947s (SENT)
I20260709 12:18:19.959151 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.011772 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.012125 25708 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:19.945406 27850 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:19.975235 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.012817 27854 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:20.013291 25751 leader_election.cc:336] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.507s (SENT)
I20260709 12:18:19.954048 27902 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:19.962777 26307 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:20.016083 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53421.7us
W20260709 12:18:19.963614 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.018082 25778 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 12:18:19.946158 27923 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.002s	sys 0.015s
W20260709 12:18:19.997279 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 353436us
I20260709 12:18:19.998080 25712 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:19.968538 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:19.968647 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:20.033046 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:20.041543 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:20.061630 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.080861 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:20.091634 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 65201us
I20260709 12:18:20.093843 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:20.131958 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 101899us
I20260709 12:18:20.131124 25713 raft_consensus.cc:2376] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2. Re-sending same reply.
W20260709 12:18:20.133262 26309 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.044s (SENT)
W20260709 12:18:20.137313 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 375271us
W20260709 12:18:20.131112 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.029547 27827 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:20.029050 27519 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:18:20.060887 27854 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:18:20.160876 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:20.161839 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.164105 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.166138 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.147238 26338 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
W20260709 12:18:20.179677 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.173813 25706 raft_consensus.cc:2376] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2. Re-sending same reply.
I20260709 12:18:20.185802 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:20.196334 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:20.179291 27966 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:18:20.177865 27902 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:20.210877 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.213143 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:20.214275 26340 raft_consensus.cc:2410] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:20.219046 27615 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:20.220587 27952 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:20.222827 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:20.223915 27690 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.001s
W20260709 12:18:20.227363 26335 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.626s	user 0.013s	sys 0.000s
I20260709 12:18:20.227878 26335 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:20.229393 27924 tablet_replica.cc:333] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:18:20.186188 27936 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:20.230986 27924 raft_consensus.cc:2243] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:20.231738 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1954) took 2193 ms. Trace:
I20260709 12:18:20.189173 27923 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:18:20.235983 27747 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:20.236454 27747 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:20.236933 27747 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.239005 27772 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:20.178915 27876 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.239522 27772 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:20.239946 27876 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.240445 27772 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.241748 27772 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:20.204649 27762 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:18:20.232198 25711 rpcz_store.cc:276] 0709 12:18:18.038393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.201490 (+1163097us) service_pool.cc:224] Handling call
0709 12:18:19.201960 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.227888 (+ 25928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.227939 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.460754 (+232815us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.728522 (+267768us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:19.728576 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.728667 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:20.231514 (+502847us) raft_consensus.cc:1685] finished
0709 12:18:20.231536 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.231672 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14582,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":26389,"thread_start_us":230703,"threads_started":1,"wal-append.queue_time_us":375107}
W20260709 12:18:20.250198 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.250507 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2025) took 2257 ms. Trace:
I20260709 12:18:20.261574 25705 rpcz_store.cc:276] 0709 12:18:17.986159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.038478 (+1052319us) service_pool.cc:224] Handling call
0709 12:18:19.038864 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.051193 (+ 12329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.051215 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.052981 (+  1766us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.592634 (+539653us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b1800361de0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:18:20.007043 (+414409us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:20.007121 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:20.007259 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:20.242629 (+235370us) raft_consensus.cc:1685] finished
0709 12:18:20.242664 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.243025 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2121,"raft.queue_time_us":617,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":19867,"spinlock_wait_cycles":8344448,"thread_start_us":414064,"threads_started":1,"wal-append.queue_time_us":1127110}
W20260709 12:18:20.270154 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 140009us
I20260709 12:18:20.273319 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:20.280081 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:20.285702 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.205945 27950 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:20.232200 27924 raft_consensus.cc:2272] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:20.247108 27568 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:20.313091 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:20.315512 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:20.319765 27961 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.626s	user 0.013s	sys 0.000s
I20260709 12:18:20.260480 27876 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:20.321549 27852 consensus_meta.cc:287] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.003s
W20260709 12:18:20.333499 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.333031 25783 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:20.322365 27932 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:20.269627 27934 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:20.336530 27962 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:20.334357 27680 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:20.304651 27923 consensus_queue.cc:1243] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:18:20.342348 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 156645us
W20260709 12:18:20.342623 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 71739.4us
I20260709 12:18:20.350910 25705 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:20.360874 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:20.361701 26339 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:20.283427 27773 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.411399 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.425252 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:20.364359 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.362885 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.493866 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:20.494241 25638 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:20.458165 25601 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.432351 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:20.497242 26339 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:20.492720 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:20.496573 25786 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:20.425709 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.493944 25636 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:20.501978 25781 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:20.342115 27785 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.375178 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.363870 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.363850 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.523941 25634 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:18:20.504686 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:20.362452 27719 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.005s
I20260709 12:18:20.544579 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:20.493269 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.491413 25631 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:20.545282 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.545745 26343 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:20.363355 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.340451 27534 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:20.497833 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:20.564749 25635 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:20.565069 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 67379.7us
I20260709 12:18:20.421028 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.566938 25637 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:20.414270 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:20.373281 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.422245 25779 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:18:20.421552 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:20.507324 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.572899 25785 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:20.573172 25636 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:20.520777 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:20.521780 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 85581.5us
W20260709 12:18:20.575276 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.571493 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.575069 25786 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
W20260709 12:18:20.575743 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:20.522478 25633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.014s	sys 0.017s
I20260709 12:18:20.525312 25781 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:18:20.577020 25633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.014s	sys 0.017s
I20260709 12:18:20.543588 25787 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:18:20.342404 27936 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:20.544605 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.579906 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:20.503521 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 143787us
I20260709 12:18:20.551018 27852 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:20.506956 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 320171us
I20260709 12:18:20.345698 27962 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:20.563089 25638 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:20.583204 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.583398 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.562919 25784 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:20.584046 25630 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:20.340945 27961 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.565815 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.568171 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:20.585150 25629 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:18:20.585251 25780 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:20.585886 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.423635 27924 ts_tablet_manager.cc:1916] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:18:20.568171 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.586669 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.587559 25632 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:20.587688 25631 raft_consensus.cc:2376] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 3. Re-sending same reply.
I20260709 12:18:20.577649 25787 pending_rounds.cc:85] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 5
I20260709 12:18:20.352059 27827 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:20.410921 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:20.508239 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.586776 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:20.604176 25705 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:20.588814 25629 raft_consensus.cc:2410] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:20.588922 25787 pending_rounds.cc:107] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:18:20.605195 25787 raft_consensus.cc:2889] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:20.574287 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.606659 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.606930 25636 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:18:20.590082 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.608623 25783 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:20.608873 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.417658 27928 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:20.599035 27850 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 12:18:20.602824 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.582814 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.611063 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:20.611647 25706 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:20.610786 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.608763 25629 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:20.357957 27934 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.614357 25713 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:20.613174 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.619416 25630 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:20.620191 25632 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:18:20.562701 27785 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.614998 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.604243 27827 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:20.622355 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.639685 25630 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:18:20.575045 27534 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.546728 27882 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:20.643731 27992 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:18:20.644062 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.652750 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.643821 25632 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:18:20.653501 25707 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:20.653818 25632 pending_rounds.cc:85] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 3
I20260709 12:18:20.654229 25632 pending_rounds.cc:107] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:18:20.654543 25632 raft_consensus.cc:2889] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:20.640033 27892 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.660781 27892 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.650345 27719 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:20.663830 27892 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:20.664492 27827 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.653359 26343 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:20.647203 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:20.644162 25631 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:18:20.643821 27651 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.682s	user 0.010s	sys 0.000s
I20260709 12:18:20.652189 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.588004 27936 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.587956 27852 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.658103 27534 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:20.659361 27882 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:20.611976 26335 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:20.673417 27651 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.677703 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:20.667160 27827 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:20.678823 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 302496us
W20260709 12:18:20.686550 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 120590us
I20260709 12:18:20.622970 27850 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:20.692173 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.699836 25707 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:20.674376 26340 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:20.711395 25601 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.716252 25782 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 6. Ops: []
I20260709 12:18:20.682308 27990 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:20.718816 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:20.702314 27955 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:20.733007 25751 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.576s (SENT)
I20260709 12:18:20.733683 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.709506 27773 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:20.753463 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 170706us
W20260709 12:18:20.760722 25751 leader_election.cc:336] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.640s (SENT)
I20260709 12:18:20.762898 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:20.724308 27993 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:20.776304 25751 leader_election.cc:336] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.640s (SENT)
I20260709 12:18:20.752617 27986 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:20.811440 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:20.812791 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.813154 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.813966 25708 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:20.759187 27959 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.831081 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 112344us
I20260709 12:18:20.762143 27719 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:20.833695 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:20.756983 27990 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:20.803823 27923 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:20.847702 25706 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:18:20.761864 27793 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.851548 27793 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.756740 27872 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:18:20.856980 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 80751.1us
I20260709 12:18:20.868120 25708 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
W20260709 12:18:20.868247 25751 leader_election.cc:336] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.643s (SENT)
I20260709 12:18:20.877592 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:20.843302 27959 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:20.885816 27959 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:20.887576 27680 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.000s
I20260709 12:18:20.888058 27955 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:20.888913 27850 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:20.857834 27928 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:20.867149 27999 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:18:20.842581 27986 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:20.898341 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.867398 27872 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.899546 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.899784 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.903179 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:20.922353 25705 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:20.922461 25712 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:20.903131 25709 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:20.904127 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.923760 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:20.904731 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:20.924855 25713 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:20.905263 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.914045 27923 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.914244 27651 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:20.913661 25751 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.736s (SENT)
I20260709 12:18:20.906592 27959 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:20.895849 27961 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:20.902997 25704 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:20.929539 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.932948 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.933707 27998 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:20.945914 26344 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:20.960996 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:20.973837 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 60193.1us
W20260709 12:18:20.974673 25751 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.737s (SENT)
I20260709 12:18:20.916579 27680 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:20.975523 25751 leader_election.cc:336] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.778s (SENT)
I20260709 12:18:20.976032 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.977558 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:20.914919 27955 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:20.933243 27872 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:20.928828 27936 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:20.932624 27934 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:20.979492 25676 connection.cc:256] RPC call timeout handler was delayed by -2.2294617499999276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 12:18:21.032999 25676 connection.cc:256] RPC call timeout handler was delayed by -2.0665617499998916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 12:18:21.033524 25676 connection.cc:256] RPC call timeout handler was delayed by -2.0260617500000535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 12:18:21.033952 25676 connection.cc:256] RPC call timeout handler was delayed by -1.9270617499998934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 12:18:21.034370 25676 connection.cc:256] RPC call timeout handler was delayed by -1.9027617500000815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
I20260709 12:18:21.035759 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:21.036608 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61107.8us
I20260709 12:18:20.944471 28010 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.041145 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 335) took 1167 ms. Trace:
I20260709 12:18:21.041540 25782 rpcz_store.cc:276] 0709 12:18:19.873673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.888192 (+ 14519us) service_pool.cc:224] Handling call
0709 12:18:19.915538 (+ 27346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.716815 (+801277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.040903 (+324088us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54001a7cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:21.041088 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840902528}
I20260709 12:18:21.039253 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:21.048561 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.049482 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:21.054220 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.054601 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.054890 25637 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:20.977873 27651 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:21.021833 27747 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.000s
W20260709 12:18:21.022644 27831 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.129s	user 0.013s	sys 0.000s
I20260709 12:18:21.026621 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:20.942466 27928 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:21.056766 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.016562 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:20.934824 27615 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 12:18:21.186801 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:20.992817 27959 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.059507 27936 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:21.063879 25631 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:20.936823 27923 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.059801 27832 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:18:21.038331 27850 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:21.109472 26343 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:18:21.063671 27932 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:21.065168 27719 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:21.064911 27534 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:21.100833 27747 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.107717 27831 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.125752 26339 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:21.204632 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.056533 27852 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:21.143467 26307 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:20.976140 27998 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.146894 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:21.213168 25637 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:21.061116 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:21.150727 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:21.168051 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.156294 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.172222 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:21.156574 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.173779 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.214761 25632 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:21.014246 27961 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:21.216454 28009 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:18:21.194852 27932 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.217492 27831 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.057204 28010 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.218359 26338 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:21.219439 25635 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:21.058827 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:21.220144 28010 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:21.186236 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 169794us
I20260709 12:18:21.220350 27747 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.221222 27831 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:21.153883 27990 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:21.223403 28002 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:18:21.198388 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.138962 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:21.226933 26342 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:21.227010 27990 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.228749 27998 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.195318 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:21.231223 25638 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:18:21.231273 27923 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.207922 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:21.205343 25634 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:21.220490 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 161767us
I20260709 12:18:21.214812 25630 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:21.210419 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 67048.3us
I20260709 12:18:21.199801 27719 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:21.234017 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:21.226794 25631 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:21.154580 26344 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:21.200241 27936 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:21.240207 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:21.245294 26342 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
W20260709 12:18:21.247002 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 194035us
W20260709 12:18:21.246379 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 189678us
I20260709 12:18:21.238514 25632 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:21.249359 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:21.206769 27928 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.238940 27832 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:21.255081 27534 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:21.256637 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:21.149669 27986 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.257566 26343 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:21.257138 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.255347 27832 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:21.271373 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 97675.9us
I20260709 12:18:21.238186 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 634) took 1418 ms. Trace:
I20260709 12:18:21.226186 27872 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.238639 27719 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:21.274739 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.272548 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.183854 27955 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.236461 27932 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:21.238370 27934 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.281035 28024 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:21.279373 27821 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:18:21.322980 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50535us
I20260709 12:18:21.243506 27831 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:21.271006 27928 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.254381 27961 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:21.319805 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45198.4us
I20260709 12:18:21.331753 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.256165 27923 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:21.243396 27618 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:21.324436 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.273499 27534 consensus_queue.cc:1243] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:21.316250 25713 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:21.273828 28001 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.258846 28007 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:21.286936 27772 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.045s	user 0.015s	sys 0.000s
I20260709 12:18:21.329654 25633 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:21.277850 27832 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.322466 27972 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.261225 27959 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:18:21.275506 27936 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.352108 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:21.355013 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 103170us
I20260709 12:18:21.321853 27821 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:21.270617 27998 consensus_queue.cc:1059] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:18:21.357232 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:21.302326 26337 rpcz_store.cc:276] 0709 12:18:19.819396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.819518 (+   122us) service_pool.cc:224] Handling call
0709 12:18:19.848008 (+ 28490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.928808 (+ 80800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.206292 (+1277484us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b540036ae54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:21.236289 (+ 29997us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55725312}
I20260709 12:18:21.343971 27987 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:21.377763 25713 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:21.361181 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.361909 26336 raft_consensus.cc:2410] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:21.361143 27772 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:21.362335 27962 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.365967 27794 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:21.366822 27827 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:21.367094 27928 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:21.369261 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.389091 27987 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.371443 27961 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.371347 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:21.372323 27990 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:21.373093 27863 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:21.378997 25707 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:18:21.404793 27794 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:21.329169 28024 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.380570 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:21.355247 27747 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:21.399662 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 177576us
I20260709 12:18:21.518460 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.520027 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.403280 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:21.521143 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 117970us
I20260709 12:18:21.399017 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:21.355485 27852 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:21.325456 27934 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:21.378845 27832 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.383960 27923 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:21.505090 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 133805us
I20260709 12:18:21.384023 28007 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:21.377166 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:21.513521 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.408787 27754 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:21.515539 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 135081us
I20260709 12:18:21.503216 27972 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.398499 27534 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:18:21.409515 28001 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 12:18:21.503750 27987 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:21.509509 27892 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:21.574314 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.574981 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:21.526973 25635 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:21.580664 25637 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:21.590827 27832 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.589777 27754 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.511709 27959 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:18:21.544046 27968 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.379s	user 0.009s	sys 0.006s
I20260709 12:18:21.517446 28024 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:21.556674 28022 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 12:18:21.602102 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 242070us
I20260709 12:18:21.524901 27961 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:21.562793 25704 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:21.555500 27989 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:21.601524 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.414978 27827 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.564038 27872 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.574155 28010 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:21.555665 27973 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.583703 27892 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:21.591869 27831 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.651888 25637 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:21.594218 27906 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:18:21.653378 27831 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:21.654629 27969 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.522s	user 0.005s	sys 0.000s
I20260709 12:18:21.655900 27831 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.608487 27972 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.629726 27793 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:21.657007 27892 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.637720 27651 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:21.658445 27892 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:21.650919 27989 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.661192 27989 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.639993 28010 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:21.662837 27934 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.642091 27534 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:18:21.525290 27863 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.640774 27928 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:21.595929 27850 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:21.664970 27850 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:21.665129 27863 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.666038 27850 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.658179 27972 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.668552 27754 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.612807 28003 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:18:21.554988 28018 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 12:18:21.609515 27959 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:21.671171 27832 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.671324 28018 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:21.592239 27901 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:21.672051 27901 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:21.671846 27959 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.673101 27901 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.606350 28025 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:21.601894 27944 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:21.675304 27944 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.675889 27901 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:21.676844 27794 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.677441 27944 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.589406 28007 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.678470 27794 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:21.680405 28007 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.680976 27785 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.566280 27821 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.678776 27754 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:21.681766 27821 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:21.682060 27754 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:21.627732 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:21.670894 28010 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.671869 28018 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:21.683439 28018 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.684820 28018 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:21.685861 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 58201.7us
I20260709 12:18:21.514240 27990 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.693842 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:21.644371 27882 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:21.611104 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:21.698030 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 87005.7us
I20260709 12:18:21.611176 27924 ts_tablet_manager.cc:1929] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:18:21.701349 27924 log.cc:1199] T 595303b3057546cbbfda4b84f8df8912 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/595303b3057546cbbfda4b84f8df8912
I20260709 12:18:21.701083 27754 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.674139 27719 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.702610 28024 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.704433 27719 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.704972 28024 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:21.706841 27961 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:21.616812 27772 consensus_queue.cc:1243] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:21.699883 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.703015 27924 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51510 (request call id 93) took 3271 ms. Trace:
I20260709 12:18:21.711009 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.716715 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.722675 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.727296 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:21.728233 25785 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:21.711627 28029 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:21.732925 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.708572 27928 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:21.734362 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.708544 27534 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.739662 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:21.743898 25781 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:21.742089 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:21.743991 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 224041us
I20260709 12:18:21.745428 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:21.746035 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.711894 27651 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.750236 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.746284 25781 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:21.748147 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:21.751391 25780 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:21.748147 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:21.749019 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:21.753620 25782 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:21.749392 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.718101 27719 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.747046 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:21.764112 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 171022us
I20260709 12:18:21.764602 25632 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:21.728932 27863 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:21.729306 27974 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.770520 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.730048 27961 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.771595 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.739295 27554 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:21.696708 27829 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:18:21.739697 27793 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.781574 27961 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.782835 27829 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:21.783259 27829 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:21.783984 27961 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:21.740296 27972 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.785646 28029 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:21.766500 27651 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.786343 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:21.779145 27990 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.779865 27827 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:21.780113 25778 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:18:21.780797 27986 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:21.782068 27793 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:21.800508 27793 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.800956 27793 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:21.802464 28029 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.804567 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:21.805378 25633 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:21.784013 27829 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.731267 27989 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:18:21.747859 27923 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:21.789529 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.790217 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:21.772411 27719 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:21.791887 27882 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:18:21.712809 27924 rpcz_store.cc:276] 0709 12:18:18.431453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.431839 (+   386us) service_pool.cc:224] Handling call
0709 12:18:18.918638 (+486799us) ts_tablet_manager.cc:1151] Deleting tablet 595303b3057546cbbfda4b84f8df8912
0709 12:18:18.920804 (+  2166us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:18:20.229202 (+1308398us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540021f554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5997ff9 00007f29ba9a5482 00007f29ba9d62d0 00007f29ba9bc83d 00007f29ba9bc7e9 00007f29ba9bc779 00007f29ba9bc741 00007f29ba9bb96d 0000000000388634 0000000000388479
0709 12:18:21.283380 (+1054178us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:21.601116 (+317736us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:21.702903 (+101787us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12700,"spinlock_wait_cycles":1466221440,"tablet-delete.queue_time_us":354921,"thread_start_us":241304,"threads_started":1}
I20260709 12:18:21.798828 27827 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:21.689064 27973 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.779044 27554 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.769227 27928 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:21.808965 25785 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:21.828418 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:21.810412 25782 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:21.807703 27829 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:21.739491 28020 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:18:21.774695 27863 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:21.883352 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 93234.4us
I20260709 12:18:21.814105 27923 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.887270 25676 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:21.888793 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:21.890890 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:21.857272 25633 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:21.792255 27986 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.809003 27974 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:21.867978 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.872895 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.908003 25636 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:18:21.875147 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.896569 25779 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:21.792089 28033 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.827129 27554 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:21.910704 25630 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:21.878396 28037 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:21.892647 28007 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:21.898365 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:21.899681 27892 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.901147 27651 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:21.829358 28025 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.893697 28024 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.857547 27852 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:21.933629 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 168056us
I20260709 12:18:21.891638 25712 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:21.935555 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.894385 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:21.907052 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 156929us
I20260709 12:18:21.936441 25633 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:21.907608 25629 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:21.815917 27827 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.857746 27989 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:21.898860 27986 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:21.895253 28043 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:21.895076 27923 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:21.902331 27719 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:21.896488 28044 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.897048 27906 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:21.894359 27863 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:21.910156 27680 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:21.939523 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.995716 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.907205 27974 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.940615 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:21.937896 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:21.942929 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:21.944439 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.941411 27989 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:22.004400 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66811.2us
W20260709 12:18:21.951177 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 56887us
I20260709 12:18:22.005865 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:22.007738 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:21.946991 27932 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.000s
I20260709 12:18:22.002969 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:21.948665 27651 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:21.878693 27821 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:18:22.019047 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.002918 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:22.003333 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.023631 25782 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:22.024565 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:21.936230 28037 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:21.963757 27944 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.951711 27990 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:22.008972 27974 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:22.025079 25782 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:21.994896 28043 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:21.939769 28024 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:21.960680 27928 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.031376 25785 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:22.041661 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:21.965173 28001 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:18:22.012648 27832 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:22.042443 27651 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:21.994082 28044 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.047720 28007 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.051893 27680 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.082310 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:22.090839 26344 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:22.062980 27934 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.094823 27934 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.095486 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.097374 27934 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:22.016342 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:21.994288 28010 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:18:21.997305 27906 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:18:22.103372 27827 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:22.064854 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.066892 26342 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 5 LEARNER]: Advancing to term 6
W20260709 12:18:22.070705 28039 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:18:22.072733 26340 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:22.049054 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.106459 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.156962 26338 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:22.064165 27989 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.160586 27989 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.161567 27754 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.046988 27990 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.166559 28007 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.169462 28007 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:22.062573 27892 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:22.170293 27892 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.171633 28031 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:22.172044 28031 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:22.172423 27892 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:22.172921 28031 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.174010 28031 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:22.175168 26336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 639) took 1913 ms (client timeout 1747 ms). Trace:
W20260709 12:18:22.175580 26336 rpcz_store.cc:269] 0709 12:18:20.261521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.279757 (+ 18236us) service_pool.cc:224] Handling call
0709 12:18:22.174984 (+1895227us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400573d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:22.175095 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485194112}
W20260709 12:18:22.108433 28046 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:18:22.073601 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:22.108733 28040 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 12:18:22.017719 27956 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:22.178563 27956 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.110376 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.179028 27956 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:21.936715 28025 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.074858 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.076610 25782 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:22.076776 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:22.044821 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 108889us
I20260709 12:18:22.165904 27990 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:22.146876 25630 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:22.200716 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.116001 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.201538 25705 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 3: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:22.187645 25780 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:22.202044 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:22.182261 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:22.202855 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.203320 25712 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:22.183415 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.103399 25784 raft_consensus.cc:2410] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:22.105594 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:22.187753 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.205099 26339 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
W20260709 12:18:22.196712 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 196143us
I20260709 12:18:22.206879 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:22.183531 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.115123 26342 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:18:22.210899 26342 pending_rounds.cc:85] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 5
I20260709 12:18:22.211249 26342 pending_rounds.cc:107] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:18:22.211539 26342 raft_consensus.cc:2889] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:22.213577 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.131440 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.137300 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.215168 25787 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.155000 25779 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:22.191370 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.216720 25708 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:22.218474 26342 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 2: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:22.184651 25709 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:22.240243 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.241698 25782 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:18:22.243122 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.244112 25782 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:22.245324 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.246130 25785 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:22.248476 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.249307 25710 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:22.252920 25711 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:22.261018 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:22.262781 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55973.2us
I20260709 12:18:22.263718 25750 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:22.264005 26339 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:22.262846 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.254029 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.271000 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.272270 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:22.273684 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:22.275413 25785 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:22.277148 25785 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:22.279330 25785 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:22.021339 27932 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.283429 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.283557 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.283890 25786 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:22.290043 25706 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 1.
I20260709 12:18:22.290170 26343 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:22.284439 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.291101 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.291417 25707 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:22.286592 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:22.293558 25713 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:22.302000 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:22.304044 25676 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:22.113502 28017 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:22.291823 25706 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:22.323819 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.324445 25706 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:22.311075 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.291687 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:22.309101 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.181000 28033 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.320331 25784 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:22.145468 27962 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.125245 28057 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.346838 28057 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.240612 28007 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.326241 25780 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:22.327430 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.138170 27747 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:22.321925 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49740.5us
I20260709 12:18:22.329679 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.353327 25779 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:22.330719 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.354907 25705 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 3: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:22.146711 28044 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:22.272451 27892 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:22.336820 27955 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:22.258985 27618 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:18:22.284246 27952 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:22.337281 28033 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.338868 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.384593 25784 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:18:22.270679 27793 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.153824 27998 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:22.320779 27934 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:22.320960 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.119442 27651 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.196694 27680 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.259286 28018 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 12:18:22.156083 27944 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:22.397136 27944 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.399371 27944 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:22.399942 28038 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.400503 28038 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.394932 25712 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:22.404037 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.406698 27998 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.408699 27998 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:22.416109 25713 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:22.329285 27832 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:22.322279 25783 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:22.349354 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 57754.6us
W20260709 12:18:22.282974 27987 consensus_meta.cc:287] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.690s	user 0.011s	sys 0.000s
I20260709 12:18:22.342262 25706 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:22.356830 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.366747 25787 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:22.425372 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.287671 27534 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:22.368355 26343 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:22.259456 27993 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.127s
I20260709 12:18:22.373533 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.375312 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.378912 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.438503 25780 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:22.439261 25705 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:22.439519 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.370997 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:22.396579 28057 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:22.323145 28017 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:22.421486 25711 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:22.425918 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:22.390612 27955 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.463443 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.428787 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.143362 27973 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:22.430974 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45568.9us
I20260709 12:18:22.436136 25706 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:22.468214 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:22.371174 27747 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:22.440778 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:22.442013 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71129.7us
I20260709 12:18:22.438691 25709 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:22.470045 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:22.470729 25709 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:22.443208 27944 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:22.386605 27719 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:22.369488 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.481587 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.450893 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:22.451711 25781 raft_consensus.cc:2368] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:22.426002 25787 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:22.454404 26335 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:22.454584 28017 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.436916 27952 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:22.456810 28018 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:22.462101 25602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.010s	sys 0.026s
W20260709 12:18:22.426472 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:22.521166 25602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.010s	sys 0.026s
I20260709 12:18:22.522950 25677 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:22.523413 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.144074 28043 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.393442 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:22.472230 25634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.052s	sys 0.102s
W20260709 12:18:22.524811 25634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.052s	sys 0.102s
I20260709 12:18:22.478057 25706 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:22.416260 27772 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:22.428725 27987 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.480214 26336 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:22.482250 26343 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:22.381902 28007 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.537067 25709 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:22.537794 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:22.538389 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:22.538923 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:22.539559 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:22.504187 25787 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:22.455047 28001 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:22.548202 25784 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:22.549094 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.550366 25787 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:22.509610 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:22.121726 28037 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.452679 28038 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:22.391571 27892 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.531190 27944 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:22.471765 25676 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:22.532274 25706 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:18:22.435906 27534 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.480909 27998 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:22.481197 27719 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.539155 25635 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:22.512204 28020 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:18:22.541394 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:22.550311 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.600010 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:22.601315 25784 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:18:22.509146 28048 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:22.602519 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:22.553643 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48171.7us
I20260709 12:18:22.603509 26343 raft_consensus.cc:2410] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:22.554576 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:22.566033 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:22.606194 25787 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:22.606935 25784 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:22.521028 28018 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.469182 27955 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.533354 28017 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.580354 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:22.615648 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.534808 27872 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:22.620533 25784 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:18:22.582559 25637 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:22.545149 27821 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:22.593448 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 121764us
I20260709 12:18:22.583716 28024 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:22.597090 26344 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:22.592803 27534 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:22.616333 26342 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:22.535190 27773 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:18:22.575652 28001 consensus_queue.cc:1243] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:18:22.591231 26338 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:22.534261 28003 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:18:22.611037 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:22.502321 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:22.658806 25783 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:22.624814 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 70342.8us
I20260709 12:18:22.665480 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:22.566450 28069 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:22.628454 25784 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:22.611744 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:22.545925 27772 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.632885 26344 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:22.706045 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:22.597023 27719 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:22.594194 27944 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.597038 27832 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:22.652397 27618 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.671838 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:22.676030 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:22.618312 28018 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:22.627285 27863 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.001s
I20260709 12:18:22.613045 27998 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:22.678450 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:22.619985 28048 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:22.648124 27793 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:22.642369 27959 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:22.622668 27952 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.705822 27936 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:22.770378 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.756588 25711 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:22.804769 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 126432us
I20260709 12:18:22.647809 27821 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.680958 28069 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:22.828629 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 205811us
I20260709 12:18:22.863580 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:22.864779 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:22.720860 27944 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:22.755931 28003 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:18:22.813902 27554 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.882s	user 0.010s	sys 0.001s
I20260709 12:18:22.814910 27832 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.815130 27932 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:22.813803 28051 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.713531 27882 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.150s
W20260709 12:18:22.877995 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 202037us
I20260709 12:18:22.715785 28044 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 12:18:22.858050 27936 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.814093 28015 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:22.704754 28038 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 12:18:22.755273 27993 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 12:18:22.876782 27793 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.887830 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:22.887880 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:22.877115 28003 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:22.711549 27973 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.151s
I20260709 12:18:22.866460 28069 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:22.804625 27618 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:22.716995 27827 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.154s
I20260709 12:18:22.872131 27680 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:22.911824 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:22.926128 25708 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.852744 27863 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.889889 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:22.918993 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58602.2us
I20260709 12:18:22.930387 25630 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:22.931471 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:22.932076 25630 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:22.933133 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:22.924204 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:22.903357 27936 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:22.918614 28003 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:22.908301 28038 consensus_queue.cc:1243] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:22.908661 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:22.917179 27754 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.001s
I20260709 12:18:22.880599 27554 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.903976 28015 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:22.917016 27987 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:22.930874 28066 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
I20260709 12:18:22.922253 27794 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:22.886169 27832 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.917457 28007 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:22.934942 27932 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:18:22.895591 28051 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:22.956532 25629 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:22.958012 27852 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:22.959494 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:23.002416 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:22.964308 28068 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:18:22.916271 28048 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.017822 25708 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
W20260709 12:18:22.955873 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66390.9us
I20260709 12:18:22.967397 25712 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:22.974295 25630 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:22.928562 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63849.4us
I20260709 12:18:22.949539 28028 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:22.980929 25783 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:22.951263 25707 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:18:22.970593 27829 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.087s	user 0.012s	sys 0.000s
I20260709 12:18:22.992071 28007 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:22.968343 28015 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:22.993407 27554 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:22.967398 27892 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:22.970950 26338 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.999253 25638 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:18:23.007052 27959 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:22.994480 27863 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:23.023149 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:23.023411 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:22.965827 27923 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.972s	user 0.011s	sys 0.000s
I20260709 12:18:23.024459 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:22.979380 28044 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.026190 25712 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:22.966760 27973 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:18:22.943166 27998 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:23.036362 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 98153.5us
I20260709 12:18:23.095662 25708 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:23.019671 25711 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:22.996031 28051 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:23.065378 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 63033.1us
I20260709 12:18:23.026158 28028 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:23.080418 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57082.4us
I20260709 12:18:23.058859 27959 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.104664 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:22.996901 27719 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.116695 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:23.113933 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:23.109246 28028 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:23.115752 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 91409.9us
I20260709 12:18:23.120380 28085 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:23.120707 28001 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.080688 28076 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:23.112905 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.087122 27892 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:23.115634 28051 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:23.115557 27959 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:23.118498 27932 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.128522 27952 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:23.100973 27923 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.097551 28044 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.156106 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:23.168689 28098 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.163552 27827 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
W20260709 12:18:23.163663 28017 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.535s	user 0.005s	sys 0.005s
W20260709 12:18:23.173379 28088 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:18:23.176157 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:23.177350 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:23.178836 25676 leader_election.cc:400] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:23.179177 25676 leader_election.cc:403] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:18:23.152642 27892 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:23.182454 25786 raft_consensus.cc:2368] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:23.134783 28001 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.152410 27932 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.169382 28098 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.189298 27934 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:23.185159 27618 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:23.180454 27952 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:23.222664 28077 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:23.184173 28007 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:18:23.216828 27832 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:23.181770 28089 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:18:23.216318 27934 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:23.219297 28091 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.218907 27892 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:23.241814 27827 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:23.247712 28031 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.073s	user 0.002s	sys 0.012s
I20260709 12:18:23.274050 28086 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:23.274822 28086 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:23.275825 28086 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.256034 27934 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:23.192137 28090 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 12:18:23.301254 28094 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:18:23.260712 27827 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:23.289960 28067 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.707s	user 0.000s	sys 0.005s
I20260709 12:18:23.295993 26339 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:18:23.315240 27952 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.317622 26339 pending_rounds.cc:85] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 6
I20260709 12:18:23.329882 26339 pending_rounds.cc:107] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:18:23.330277 26339 raft_consensus.cc:2889] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:23.322027 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:23.322561 28095 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.152s
I20260709 12:18:23.350531 28051 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:23.352180 27872 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:23.352687 27872 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:23.289250 27719 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:23.295634 28033 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:23.353199 28099 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:18:23.371110 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:23.375413 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:23.376070 25705 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:23.396765 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 218347us
I20260709 12:18:23.397957 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:23.384788 27892 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.384934 27989 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:23.385231 28007 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.260324 28091 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.400257 27932 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:23.470886 26342 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:23.406989 28017 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:23.478664 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:23.502524 25638 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:23.467062 27832 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:23.505802 26342 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:18:23.506263 26342 pending_rounds.cc:85] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 6
W20260709 12:18:23.468537 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 70687.2us
I20260709 12:18:23.506847 26342 pending_rounds.cc:107] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:18:23.507319 26342 raft_consensus.cc:2889] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:23.509637 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:23.512022 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 169583us
I20260709 12:18:23.462406 27989 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:23.516615 27974 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.463s	user 0.010s	sys 0.021s
I20260709 12:18:23.517072 27974 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.518424 25633 raft_consensus.cc:2368] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4.
W20260709 12:18:23.519572 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 259733us
I20260709 12:18:23.505642 27832 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.521346 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:23.521482 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:23.525835 26342 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:23.526823 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:23.528345 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:23.500427 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1961) took 1465 ms. Trace:
W20260709 12:18:23.512423 28081 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:18:23.535315 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:23.665876 25630 raft_consensus.cc:2410] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:23.528051 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:23.532673 25705 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:23.694635 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:23.699432 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:23.699335 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:23.710325 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:23.714061 25705 raft_consensus.cc:2410] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:23.715728 28101 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:18:23.719650 28106 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.720438 28106 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.723426 28106 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:23.724958 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:23.464417 27892 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.544997 28025 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:23.529522 27832 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:23.523615 28001 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:23.407450 27827 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.736681 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:23.538321 26335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 450) took 1459 ms (client timeout 1760 ms). Trace:
I20260709 12:18:23.660813 27998 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:23.539137 27932 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:23.471771 28007 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:23.753414 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 204914us
I20260709 12:18:23.754807 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:23.756213 25708 raft_consensus.cc:2410] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:23.757431 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:23.661057 28089 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:23.531270 28051 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:23.762066 26342 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
W20260709 12:18:23.747664 28043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.021s	sys 0.046s
W20260709 12:18:23.790012 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58651us
I20260709 12:18:23.507669 28017 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.725906 28033 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:23.776582 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:23.738031 27990 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.785312 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.757530 28101 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:18:23.762758 27986 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.801s	user 0.014s	sys 0.018s
W20260709 12:18:23.796311 25750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 2.133s (SENT) [suppressed 30 similar messages]
I20260709 12:18:23.759835 27998 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.760480 28106 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.781538 27944 consensus_meta.cc:287] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.901s	user 0.010s	sys 0.004s
W20260709 12:18:23.790478 28043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.021s	sys 0.047s
I20260709 12:18:23.777434 27827 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:23.809610 27794 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.822s	user 0.010s	sys 0.000s
I20260709 12:18:23.813107 27986 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.812803 28109 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:23.815604 28101 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:23.806401 28098 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:23.860213 28110 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:23.860576 28065 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:18:23.861626 28111 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:18:23.902967 28065 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:23.904058 28065 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:23.815024 27956 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.635s	user 0.013s	sys 0.005s
I20260709 12:18:23.907042 25636 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:23.868724 27934 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 12:18:23.871109 28091 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:23.899664 27962 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.532s	user 0.012s	sys 0.002s
I20260709 12:18:23.768206 28001 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:18:23.923405 25779 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
W20260709 12:18:23.811088 27936 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.857s	user 0.010s	sys 0.003s
W20260709 12:18:23.905633 27793 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.989s	user 0.013s	sys 0.000s
I20260709 12:18:23.924378 27936 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.925074 27793 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:23.887017 27794 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:23.845351 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:23.927093 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1520) took 2097 ms. Trace:
I20260709 12:18:23.927799 26342 raft_consensus.cc:2410] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:18:23.928553 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1498) took 1632 ms (client timeout 1812 ms). Trace:
I20260709 12:18:23.928913 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:23.929096 25786 rpcz_store.cc:269] 0709 12:18:22.296513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.310845 (+ 14332us) service_pool.cc:224] Handling call
0709 12:18:23.928331 (+1617486us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54001a63d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:23.928490 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1969894144}
I20260709 12:18:23.929577 26342 raft_consensus.cc:2376] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 3. Re-sending same reply.
I20260709 12:18:23.811033 28033 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:23.816958 27534 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.139s	user 0.013s	sys 0.004s
W20260709 12:18:23.819020 27747 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.313s	user 0.002s	sys 0.014s
I20260709 12:18:23.803408 28089 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.820143 27998 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:23.756202 27989 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:23.759550 26335 rpcz_store.cc:269] 0709 12:18:22.079408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.080643 (+  1235us) service_pool.cc:224] Handling call
0709 12:18:23.538113 (+1457470us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540061d0d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:23.538253 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632455040}
I20260709 12:18:23.780979 27932 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:23.916558 26338 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.826s	user 0.000s	sys 0.013s
W20260709 12:18:23.918987 28003 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.962s	user 0.006s	sys 0.006s
I20260709 12:18:23.543427 25778 rpcz_store.cc:276] 0709 12:18:22.004843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.147568 (+142725us) service_pool.cc:224] Handling call
0709 12:18:22.147918 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.148312 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.148353 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.148850 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.435316 (+286466us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:22.435501 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.453826 (+ 18325us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.286269 (+832443us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:23.411664 (+125395us) raft_consensus.cc:1685] finished
0709 12:18:23.416345 (+  4681us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.470217 (+ 53872us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1233,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":13491072,"thread_start_us":245751,"threads_started":1,"wal-append.queue_time_us":375519}
I20260709 12:18:23.919682 28101 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:23.815028 28069 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.879s	user 0.009s	sys 0.020s
W20260709 12:18:23.953150 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.927552 25637 rpcz_store.cc:276] 0709 12:18:21.829698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.867213 (+ 37515us) service_pool.cc:224] Handling call
0709 12:18:21.881489 (+ 14276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.621992 (+740503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.926844 (+1304852us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400300cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:23.927035 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611698560}
I20260709 12:18:23.931483 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:23.927131 27936 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:23.851047 27944 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:23.936053 28028 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.804s	user 0.010s	sys 0.000s
I20260709 12:18:23.936211 27934 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.937470 27821 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:23.811192 27990 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.937129 28051 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.939482 28007 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:23.940665 28086 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:23.941387 28015 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.877s	user 0.013s	sys 0.002s
I20260709 12:18:24.004180 25710 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:18:23.881989 28060 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.901219 27974 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:23.857496 28106 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:23.948413 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 172096us
W20260709 12:18:23.936113 27955 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.296s	user 0.012s	sys 0.000s
W20260709 12:18:23.949677 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 158778us
I20260709 12:18:23.951529 28003 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:23.953145 27989 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:23.964977 28038 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.970741 27793 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.975327 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:24.002144 28017 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:23.949108 26338 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:24.005597 25637 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:23.947032 27534 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.948302 27747 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:24.001714 28077 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.005s
I20260709 12:18:23.905200 27956 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.034806 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:24.021647 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:24.037863 28121 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:23.894456 28109 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.038764 28121 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:24.038959 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 292) took 2152 ms. Trace:
I20260709 12:18:24.039382 28121 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.040748 28109 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:24.039495 25704 rpcz_store.cc:276] 0709 12:18:21.886442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.904035 (+ 17593us) service_pool.cc:224] Handling call
0709 12:18:21.904408 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:21.998362 (+ 93954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.006131 (+  7769us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.059622 (+ 53491us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.531054 (+471432us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:22.575104 (+ 44050us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.584571 (+  9467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.038403 (+1453832us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5400105254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:24.038476 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.038566 (+    90us) raft_consensus.cc:1685] finished
0709 12:18:24.038696 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.038883 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10045,"prepare.queue_time_us":35052,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":33701,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":21355,"spinlock_wait_cycles":78930816,"thread_start_us":420968,"threads_started":1,"wal-append.queue_time_us":442926,"wal-append.run_cpu_time_us":3647,"wal-append.run_wall_time_us":1462249}
I20260709 12:18:24.042398 28121 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.018438 27892 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.005127 27821 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:23.976977 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:24.053021 25783 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:24.006040 28113 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:24.048883 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:24.052214 25785 raft_consensus.cc:2368] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 2. Current term is 3.
I20260709 12:18:23.952701 28069 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:23.962026 28028 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.055948 25635 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:24.072793 26341 raft_consensus.cc:2368] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:24.046583 28077 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:24.083941 25712 raft_consensus.cc:2410] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:24.033812 28001 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.022147 27955 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.034020 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 659) took 1222 ms. Trace:
I20260709 12:18:24.061008 26335 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:24.017702 28015 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.029978 27793 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.029666 28038 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.096491 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:24.099182 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:24.063727 28113 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:24.125264 25783 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:24.125447 28113 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:24.103263 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 70067.8us
I20260709 12:18:24.119210 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1855) took 1177 ms. Trace:
I20260709 12:18:24.090554 27986 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.025799 28060 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.072114 27821 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:24.268019 25784 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:18:24.130549 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:24.234175 25637 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:24.281109 25783 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
W20260709 12:18:24.149003 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.285898 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.293171 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:24.296325 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 253103us
W20260709 12:18:24.130959 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.299856 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.302824 28112 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.320s	user 0.000s	sys 0.002s
I20260709 12:18:24.304484 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 456) took 1642 ms. Trace:
I20260709 12:18:24.305001 26344 rpcz_store.cc:276] 0709 12:18:22.662368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.706414 (+ 44046us) service_pool.cc:224] Handling call
0709 12:18:22.706782 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.710570 (+  3788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.710619 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:23.511731 (+801112us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b2c0009f618. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:18:23.512134 (+   403us) log.cc:844] Serialized 26 byte log entry
0709 12:18:23.877423 (+365289us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.877471 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.877557 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.304046 (+426489us) raft_consensus.cc:1685] finished
0709 12:18:24.304081 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.304396 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3330,"raft.queue_time_us":50935,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8456,"spinlock_wait_cycles":3135360,"thread_start_us":364493,"threads_started":1,"wal-append.queue_time_us":375509}
I20260709 12:18:24.101641 28118 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:24.101187 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:24.120795 28018 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.309s	user 0.009s	sys 0.005s
I20260709 12:18:24.101715 28017 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:24.374221 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 83146.8us
I20260709 12:18:24.279846 25638 rpcz_store.cc:276] 0709 12:18:22.941668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.746402 (+804734us) service_pool.cc:224] Handling call
0709 12:18:23.750088 (+  3686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.750287 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:24.021874 (+271587us) log.cc:844] Serialized 14 byte log entry
0709 12:18:24.118776 (+ 96902us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:24.118842 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:24.118897 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.118983 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.119137 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8967,"prepare.queue_time_us":283503,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":9106,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":79934,"thread_start_us":323505,"threads_started":2}
I20260709 12:18:24.407680 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:24.383406 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:24.385897 25704 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
W20260709 12:18:24.453687 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 78098.2us
I20260709 12:18:24.098501 26338 rpcz_store.cc:276] 0709 12:18:22.812099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.855546 (+ 43447us) service_pool.cc:224] Handling call
0709 12:18:24.032283 (+1176737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.466686 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 612) took 2086 ms. Trace:
I20260709 12:18:24.200145 26340 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:18:24.477804 27534 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.121533 27618 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.260s
I20260709 12:18:24.378650 27719 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:24.478013 27747 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.505980 27772 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:24.462750 27793 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.283349 27794 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:24.395085 27832 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.001s
W20260709 12:18:24.478837 27852 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.453s	user 0.019s	sys 0.014s
W20260709 12:18:24.436110 27872 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.083s	user 0.009s	sys 0.006s
I20260709 12:18:24.456794 27934 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.375488 27936 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:24.371793 27952 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:24.495205 27956 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:24.120321 27987 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:24.460428 27990 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:24.097879 28001 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.478435 28003 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:24.452570 28007 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.436815 28015 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.381187 28017 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.382123 28018 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.392678 28053 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:18:24.494833 28069 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:24.836966 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:24.200662 28075 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.207s	user 0.003s	sys 0.009s
I20260709 12:18:24.441970 28089 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:24.256778 28087 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.103s	user 0.000s	sys 0.013s
I20260709 12:18:24.546705 28084 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.843184 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2064) took 2321 ms. Trace:
W20260709 12:18:24.563678 28080 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.550s	user 0.007s	sys 0.001s
I20260709 12:18:24.317104 28098 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:24.846076 28098 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:24.272931 28097 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.022s	user 0.014s	sys 0.065s
I20260709 12:18:24.528430 28060 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:24.854508 27794 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.544970 28072 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.574129 28093 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.390s	user 0.002s	sys 0.002s
I20260709 12:18:24.832803 28038 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:24.832967 28076 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:24.235126 28104 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.333s	user 0.002s	sys 0.000s
I20260709 12:18:24.781701 27821 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:24.834733 27829 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.842562 28084 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.286074 28091 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 12:18:24.850003 27719 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.851083 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:24.851902 27832 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:24.868391 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2002) took 1495 ms. Trace:
I20260709 12:18:24.868214 28072 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.869886 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2069) took 2035 ms. Trace:
I20260709 12:18:24.381507 28108 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.996423 28090 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.997347 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:24.997435 28090 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:24.489650 28105 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 12:18:24.540478 28140 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.997785 28108 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:24.393301 28106 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:24.320294 28109 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:24.388180 28118 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:24.502458 28120 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
W20260709 12:18:24.501484 28123 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:18:24.472572 27944 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:24.773444 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.966068 27829 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.778992 27892 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.564457 27998 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.002s
I20260709 12:18:24.911690 27852 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:24.912653 26308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.034s	sys 0.032s
I20260709 12:18:24.912153 27952 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.911939 28007 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:24.913686 27793 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.914166 26310 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.023s	sys 0.046s
I20260709 12:18:24.914674 27872 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:24.912318 28137 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.916005 27936 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:24.916997 25705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.007s	sys 0.041s
W20260709 12:18:24.574661 27959 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.422s	user 0.002s	sys 0.017s
I20260709 12:18:24.918635 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:24.843884 25706 rpcz_store.cc:276] 0709 12:18:22.521921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.590700 (+ 68779us) service_pool.cc:224] Handling call
0709 12:18:22.591005 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.811342 (+220337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.811373 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.811831 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.967338 (+155507us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:22.967374 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.967438 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.256083 (+1288645us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:24.842370 (+586287us) raft_consensus.cc:1685] finished
0709 12:18:24.842415 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.842869 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9860,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":17477,"spinlock_wait_cycles":6187392,"thread_start_us":301502,"threads_started":2,"wal-append.queue_time_us":231704}
I20260709 12:18:24.919508 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:24.920652 28028 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:24.921679 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2004) took 1400 ms. Trace:
W20260709 12:18:24.922173 25784 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.008s	sys 0.030s
W20260709 12:18:24.545912 27989 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.002s
I20260709 12:18:24.966037 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1532) took 2406 ms. Trace:
W20260709 12:18:24.966698 28051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.007s	sys 0.013s
W20260709 12:18:24.968592 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1499) took 1443 ms (client timeout 1684 ms). Trace:
W20260709 12:18:24.969129 25786 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.011s	sys 0.022s
I20260709 12:18:24.968997 28084 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:24.581418 28125 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:18:24.980718 25710 rpcz_store.cc:276] 0709 12:18:23.372462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.372612 (+   150us) service_pool.cc:224] Handling call
0709 12:18:23.373031 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.004852 (+631821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.867620 (+862768us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b5400104354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:24.868175 (+   555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450714496}
I20260709 12:18:24.995141 27821 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:24.995533 26307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.039s
W20260709 12:18:24.995785 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.997062 25780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1494) took 2701 ms (client timeout 1981 ms). Trace:
W20260709 12:18:25.000888 25783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.000s	sys 0.042s
I20260709 12:18:24.999388 28140 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.003796 27955 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:25.004385 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.017s	sys 0.013s
I20260709 12:18:25.005932 25750 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:24.549058 27923 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:24.918637 26339 rpcz_store.cc:276] 0709 12:18:22.380432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.380761 (+   329us) service_pool.cc:224] Handling call
0709 12:18:22.504431 (+123670us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:22.504633 (+   202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.330874 (+826241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.466315 (+1135441us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b540044c0d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:24.466591 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832896512}
I20260709 12:18:24.576086 28133 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.566987 28113 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:25.088142 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 351166us
I20260709 12:18:24.981521 25707 rpcz_store.cc:276] 0709 12:18:22.820970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.024282 (+203312us) service_pool.cc:224] Handling call
0709 12:18:23.024706 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.027109 (+  2403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.027168 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:23.027719 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:18:23.129940 (+102221us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.129986 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.130064 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.096767 (+966703us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:24.855998 (+759231us) raft_consensus.cc:1685] finished
0709 12:18:24.856054 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.856512 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16708,"raft.queue_time_us":2033,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":23474,"spinlock_wait_cycles":17656704,"thread_start_us":101794,"threads_started":1,"wal-append.queue_time_us":186099}
I20260709 12:18:25.002090 25704 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
W20260709 12:18:25.248207 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.017s	sys 0.016s
W20260709 12:18:25.218799 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1495) took 2923 ms (client timeout 1559 ms). Trace:
W20260709 12:18:25.271916 26310 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.026s	sys 0.054s
W20260709 12:18:25.248732 26307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.011s	sys 0.040s
W20260709 12:18:25.248792 25783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.002s	sys 0.050s
W20260709 12:18:25.249756 25784 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.011s	sys 0.033s
W20260709 12:18:25.259505 26308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.035s	sys 0.057s
I20260709 12:18:25.244077 25712 rpcz_store.cc:276] 0709 12:18:23.520937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.521064 (+   127us) service_pool.cc:224] Handling call
0709 12:18:24.921079 (+1400015us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54002ec2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:24.921394 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285096704}
I20260709 12:18:25.217671 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:25.291900 25705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.012s	sys 0.046s
I20260709 12:18:25.243983 25629 rpcz_store.cc:276] 0709 12:18:22.547674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.548183 (+   509us) service_pool.cc:224] Handling call
0709 12:18:22.548485 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.986100 (+437615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.852730 (+1866630us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54003007d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:24.954516 (+101786us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933756288}
I20260709 12:18:25.272578 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 506) took 1857 ms. Trace:
W20260709 12:18:25.295854 25786 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.018s	sys 0.024s
I20260709 12:18:25.103160 28109 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:25.326911 25632 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:25.327943 28077 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:25.328260 27872 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.328894 27829 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.328822 28077 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:25.331096 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 325439us
I20260709 12:18:25.332240 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:25.334481 28077 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.335811 28018 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:25.243878 25780 rpcz_store.cc:269] 0709 12:18:22.295366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.440538 (+145172us) service_pool.cc:224] Handling call
0709 12:18:24.996398 (+2555860us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b5400353c54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:24.996780 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491139968}
I20260709 12:18:25.090874 27832 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.096048 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:25.274772 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.104664 28076 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.356748 27892 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.359086 27821 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:25.246170 25779 rpcz_store.cc:269] 0709 12:18:23.525464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.525610 (+   146us) service_pool.cc:224] Handling call
0709 12:18:24.968117 (+1442507us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54002b5554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:24.968371 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513141888}
W20260709 12:18:25.343590 28101 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.000s	sys 0.029s
I20260709 12:18:25.098461 28001 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.343994 28033 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:25.344151 28098 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:25.344949 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.344995 28109 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:25.345587 28053 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:25.346275 27928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.467s	user 0.043s	sys 0.075s
I20260709 12:18:25.347105 27923 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:25.088586 27719 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.336997 25708 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:25.109824 28106 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.108676 28121 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:25.109264 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 734892us
W20260709 12:18:25.249886 28051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.010s	sys 0.016s
I20260709 12:18:25.325996 25638 rpcz_store.cc:276] 0709 12:18:23.387978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.008967 (+1620989us) service_pool.cc:224] Handling call
0709 12:18:25.009616 (+   649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.010282 (+   666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.010318 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.010390 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.010430 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.010558 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.245530 (+234972us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19270,"raft.queue_time_us":1030,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":67340}
W20260709 12:18:25.273799 25785 rpcz_store.cc:269] 0709 12:18:22.295631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.356600 (+ 60969us) service_pool.cc:224] Handling call
0709 12:18:25.208728 (+2852128us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54001a77d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.218551 (+  9823us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3742502784}
I20260709 12:18:25.289652 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 297) took 1224 ms. Trace:
W20260709 12:18:25.443182 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.363235 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:25.488341 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2007) took 1763 ms. Trace:
I20260709 12:18:25.562095 25630 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:25.362871 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 1973) took 1043 ms. Trace:
W20260709 12:18:25.673758 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 195090us
W20260709 12:18:25.361668 28101 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.000s	sys 0.030s
I20260709 12:18:25.677366 25787 rpcz_store.cc:276] 0709 12:18:24.319557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.216973 (+897416us) service_pool.cc:224] Handling call
0709 12:18:25.362757 (+145784us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:25.576942 25637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1861) took 1608 ms (client timeout 1921 ms). Trace:
W20260709 12:18:25.577311 25633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1848) took 3637 ms (client timeout 1532 ms). Trace:
I20260709 12:18:25.108979 28089 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:25.530203 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1507) took 1551 ms. Trace:
I20260709 12:18:25.581260 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:25.368091 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:25.664417 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 507) took 2264 ms. Trace:
I20260709 12:18:25.109567 27934 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:25.578083 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 1996) took 3060 ms. Trace:
I20260709 12:18:25.689134 25629 rpcz_store.cc:276] 0709 12:18:23.399544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.527945 (+2128401us) service_pool.cc:224] Handling call
0709 12:18:25.528597 (+   652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.529060 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.529097 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.529170 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.529248 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.529320 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.664325 (+135005us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:25.694922 25710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2013) took 1416 ms (client timeout 1622 ms). Trace:
W20260709 12:18:25.695416 25710 rpcz_store.cc:269] 0709 12:18:24.279356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.362751 (+1083395us) service_pool.cc:224] Handling call
0709 12:18:25.694847 (+332096us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.675927 25704 rpcz_store.cc:276] 0709 12:18:24.037656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.996615 (+958959us) service_pool.cc:224] Handling call
0709 12:18:25.248651 (+252036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.697249 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:25.698312 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2010) took 1420 ms. Trace:
I20260709 12:18:25.698691 25704 rpcz_store.cc:276] 0709 12:18:24.277552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.696775 (+1419223us) service_pool.cc:224] Handling call
0709 12:18:25.698227 (+  1452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.368743 27998 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.700765 27932 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:25.702696 25713 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:25.702723 27923 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:25.703481 25713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2086) took 1383 ms (client timeout 1728 ms). Trace:
W20260709 12:18:25.703977 25713 rpcz_store.cc:269] 0709 12:18:24.320105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.580643 (+1260538us) service_pool.cc:224] Handling call
0709 12:18:25.703394 (+122751us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:25.704571 25634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1525) took 3167 ms (client timeout 2083 ms). Trace:
I20260709 12:18:25.705423 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 356) took 1416 ms. Trace:
I20260709 12:18:25.704600 27923 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.706777 28044 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.707693 28113 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:25.705080 25634 rpcz_store.cc:269] 0709 12:18:22.536738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.536878 (+   140us) service_pool.cc:224] Handling call
0709 12:18:25.704168 (+3167290us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b54002f8354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.704373 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6697478528}
I20260709 12:18:25.709192 28090 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.705274 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:25.585086 28115 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.726s	user 0.000s	sys 0.002s
W20260709 12:18:25.687683 25633 rpcz_store.cc:269] 0709 12:18:21.939807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.942618 (+  2811us) service_pool.cc:224] Handling call
0709 12:18:25.576718 (+3634100us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540020b554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.577032 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3491351424}
W20260709 12:18:25.578562 28062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.026s	sys 0.004s
W20260709 12:18:25.685925 25637 rpcz_store.cc:269] 0709 12:18:23.968245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.059140 (+ 90895us) service_pool.cc:224] Handling call
0709 12:18:25.576328 (+1517188us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b54003011d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.576675 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39547776}
I20260709 12:18:25.692131 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:25.712301 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 201756us
W20260709 12:18:25.684494 27928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.874s	user 0.051s	sys 0.085s
I20260709 12:18:25.575980 27936 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.720137 27998 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.732028 27959 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.734715 28108 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:25.735186 27794 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:25.736810 26341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 609) took 3527 ms (client timeout 2046 ms). Trace:
I20260709 12:18:25.736956 27952 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:25.737325 26341 rpcz_store.cc:269] 0709 12:18:22.209185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.239948 (+ 30763us) service_pool.cc:224] Handling call
0709 12:18:25.736175 (+3496227us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54000a2b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.736533 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4186775808}
I20260709 12:18:25.739686 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 501) took 2792 ms. Trace:
I20260709 12:18:25.740162 25632 rpcz_store.cc:276] 0709 12:18:22.946927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.772788 (+825861us) service_pool.cc:224] Handling call
0709 12:18:23.803558 (+ 30770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.327493 (+1523935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.739313 (+411820us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54001c5cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:25.739598 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3502445440}
I20260709 12:18:25.741410 25603 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:22.308380 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 595303b3057546cbbfda4b84f8df8912 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:18:25.746788 27989 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:25.747255 27989 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:25.747599 25521 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 3.54676e+06us
I20260709 12:18:25.720858 25713 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:25.720983 25707 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:25.722292 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 508) took 1420 ms. Trace:
I20260709 12:18:25.390838 27821 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.378782 28072 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.705922 25781 rpcz_store.cc:276] 0709 12:18:24.288788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.704191 (+1415403us) service_pool.cc:224] Handling call
0709 12:18:25.704681 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.705057 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.705082 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.705144 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.705177 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.705245 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.705350 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.722951 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:25.450686 28076 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.747714 27989 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:25.752987 25532 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 5, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 5 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:25.808562 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:25.713702 25782 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
W20260709 12:18:25.773283 25708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2078) took 1817 ms (client timeout 1856 ms). Trace:
I20260709 12:18:25.584688 28118 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:25.716153 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.021s	sys 0.049s
I20260709 12:18:25.821945 25629 rpcz_store.cc:276] 0709 12:18:24.302073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.715816 (+1413743us) service_pool.cc:224] Handling call
0709 12:18:25.721355 (+  5539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.721850 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.721875 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.721950 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.721988 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.722063 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.722178 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.767805 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:25.821424 25707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 294) took 1784 ms (client timeout 1945 ms). Trace:
W20260709 12:18:25.827680 25707 rpcz_store.cc:269] 0709 12:18:24.036954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.517875 (+1480921us) service_pool.cc:224] Handling call
0709 12:18:25.821355 (+303480us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.689604 25709 rpcz_store.cc:276] 0709 12:18:22.517827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.539591 (+ 21764us) service_pool.cc:224] Handling call
0709 12:18:22.671085 (+131494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.671258 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.671286 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:22.671323 (+    37us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:22.671351 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.671411 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.577603 (+2906192us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400203d50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.577847 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301790464}
W20260709 12:18:25.823248 25708 rpcz_store.cc:269] 0709 12:18:23.955791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.918692 (+962901us) service_pool.cc:224] Handling call
0709 12:18:25.773053 (+854361us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b54000730d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.773211 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208917248}
I20260709 12:18:25.369814 28065 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:25.756773 25636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1847) took 3817 ms (client timeout 2133 ms). Trace:
I20260709 12:18:25.824765 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 1972) took 1505 ms. Trace:
I20260709 12:18:25.834795 25780 rpcz_store.cc:276] 0709 12:18:24.319294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.367571 (+1048277us) service_pool.cc:224] Handling call
0709 12:18:25.824698 (+457127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.757050 25631 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:18:25.811439 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1970) took 1492 ms. Trace:
I20260709 12:18:25.757961 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 498) took 3173 ms. Trace:
I20260709 12:18:25.676556 25705 rpcz_store.cc:276] 0709 12:18:23.724451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.267485 (+543034us) service_pool.cc:224] Handling call
0709 12:18:24.267795 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.268231 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:18:24.268267 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.275316 (+  7049us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.371665 (+1096349us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:25.468163 (+ 96498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.468255 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.468318 (+    63us) raft_consensus.cc:1685] finished
0709 12:18:25.468331 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.488216 (+ 19885us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":361,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":109,"thread_start_us":1080039,"threads_started":1,"wal-append.queue_time_us":957110}
W20260709 12:18:25.801055 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60141.3us
W20260709 12:18:25.809407 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.833262 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.377s	user 0.022s	sys 0.050s
I20260709 12:18:25.833451 25634 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:25.836180 25635 rpcz_store.cc:276] 0709 12:18:22.584752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.657584 (+ 72832us) service_pool.cc:224] Handling call
0709 12:18:22.658003 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.082080 (+1424077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.757396 (+1675316us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54001d79d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:25.757769 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3214629632}
I20260709 12:18:25.840914 27934 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:25.829947 25782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 360) took 1451 ms (client timeout 1546 ms). Trace:
I20260709 12:18:25.836983 25779 rpcz_store.cc:276] 0709 12:18:24.318738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.717363 (+1398625us) service_pool.cc:224] Handling call
0709 12:18:25.810456 (+ 93093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.810791 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.810827 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.810916 (+    89us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:25.810961 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.811033 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.811172 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:25.844841 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 160009us
I20260709 12:18:25.845731 25713 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:25.776319 28031 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:25.805290 28062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.302s	user 0.027s	sys 0.012s
W20260709 12:18:25.843014 25782 rpcz_store.cc:269] 0709 12:18:24.372555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.095462 (+722907us) service_pool.cc:224] Handling call
0709 12:18:25.824288 (+728826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.105023 27987 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:25.703984 28140 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.727048 27892 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:25.861052 25713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2087) took 1479 ms (client timeout 1954 ms). Trace:
I20260709 12:18:25.797621 28017 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:25.834705 25636 rpcz_store.cc:269] 0709 12:18:21.939479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.940240 (+   761us) service_pool.cc:224] Handling call
0709 12:18:25.756265 (+3816025us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b540018ddd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:25.756552 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4386142080}
I20260709 12:18:25.849458 28031 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.775944 27882 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:25.879879 27882 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.880707 25676 leader_election.cc:400] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:25.881083 25676 leader_election.cc:403] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:25.882521 27882 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.865429 27962 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:25.882956 27892 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:18:25.770944 28007 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.767388 28099 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:25.869215 25713 rpcz_store.cc:269] 0709 12:18:24.367687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.704940 (+1337253us) service_pool.cc:224] Handling call
0709 12:18:25.846675 (+141735us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.868265 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:25.365685 28137 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.892582 25634 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:25.894958 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:25.875674 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:25.541224 27793 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.794507 27959 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:25.787249 27952 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.579043 28133 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:25.866366 28108 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:25.870210 28138 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:25.878608 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1503) took 1740 ms. Trace:
I20260709 12:18:25.879014 28031 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.688690 25786 rpcz_store.cc:276] 0709 12:18:23.978543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.026745 (+ 48202us) service_pool.cc:224] Handling call
0709 12:18:24.027135 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.027597 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.027644 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.028233 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.284871 (+256638us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:25.360571 (+1075700us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.529687 (+169116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.529793 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.529876 (+    83us) raft_consensus.cc:1685] finished
0709 12:18:25.529889 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.530107 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78066,"prepare.queue_time_us":105845,"raft.queue_time_us":79452,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":4699,"spinlock_wait_cycles":25524608,"thread_start_us":1194428,"threads_started":2,"wal-append.queue_time_us":418431}
I20260709 12:18:25.887184 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:25.904035 27973 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.904083 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:25.801414 28089 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:25.901823 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 67212.6us
I20260709 12:18:25.906150 25707 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:25.872821 27936 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.780211 28106 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:25.906657 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 1974) took 1586 ms. Trace:
I20260709 12:18:25.905149 25712 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:18:26.187427 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.115950 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 359) took 1747 ms. Trace:
I20260709 12:18:25.978901 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:25.980125 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.069340 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:25.913858 25706 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:26.204519 25708 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
W20260709 12:18:26.205094 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 299) took 2167 ms (client timeout 1888 ms). Trace:
W20260709 12:18:26.205514 25706 rpcz_store.cc:269] 0709 12:18:24.038130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.808074 (+1769944us) service_pool.cc:224] Handling call
0709 12:18:26.205016 (+396942us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.935222 25782 pending_rounds.cc:85] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 4
I20260709 12:18:26.113231 25709 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:25.975355 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.201207 25785 rpcz_store.cc:276] 0709 12:18:24.319843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.694200 (+1374357us) service_pool.cc:224] Handling call
0709 12:18:25.860751 (+166551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.897900 (+ 37149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.897932 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.906185 (+  8253us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:25.906262 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.906368 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.906570 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.955726 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:26.159083 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.200786 25780 rpcz_store.cc:276] 0709 12:18:24.368666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.887239 (+1518573us) service_pool.cc:224] Handling call
0709 12:18:25.927508 (+ 40269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.109292 (+181784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.109358 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.112717 (+  3359us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.115604 (+  2887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.115709 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.115873 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.049546 25638 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:18:26.199066 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 127905us
W20260709 12:18:25.989183 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 97440.9us
W20260709 12:18:25.972801 28150 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 12:18:25.923740 25751 leader_election.cc:336] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.532s (SENT)
I20260709 12:18:26.209576 25782 pending_rounds.cc:107] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 12:18:26.227941 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1860) took 2248 ms. Trace:
I20260709 12:18:26.238629 25638 rpcz_store.cc:276] 0709 12:18:23.967931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.719198 (+1751267us) service_pool.cc:224] Handling call
0709 12:18:25.873968 (+154770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.215675 (+341707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.215912 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.232896 25782 raft_consensus.cc:2889] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:25.971197 28141 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.118847 28140 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.922972 27962 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:25.973486 28091 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:25.974782 28099 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.277153 28137 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.276459 28138 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.277782 27973 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.278856 28099 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:26.279098 26309 connection.cc:256] RPC call timeout handler was delayed by -1.1951026079998428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 12:18:26.280308 27973 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.281253 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:26.281772 28156 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:18:26.282043 26339 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:26.282344 28089 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:25.971689 28118 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.283550 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 451) took 3679 ms. Trace:
W20260709 12:18:26.259367 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 371475us
I20260709 12:18:26.274958 28001 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:26.283625 28118 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.285285 28001 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.285849 28001 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:26.287194 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1853) took 3356 ms. Trace:
I20260709 12:18:26.287617 25630 rpcz_store.cc:276] 0709 12:18:22.930771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.734548 (+803777us) service_pool.cc:224] Handling call
0709 12:18:23.734860 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.685878 (+1951018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.286865 (+600987us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54003016d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:26.287093 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3764523776}
I20260709 12:18:26.288923 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.283686 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.275921 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2082) took 2242 ms. Trace:
I20260709 12:18:26.290879 28140 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.291352 26338 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:26.129333 27952 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.294278 27952 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.294724 28148 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.275715 28123 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:18:26.284273 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.295967 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.296207 26337 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:26.234176 27882 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.297262 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.297459 28138 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.276808 25708 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:26.266330 25632 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:26.290231 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:26.229688 28028 raft_consensus.cc:670] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } [suppressed 5 similar messages]
I20260709 12:18:26.291292 25707 rpcz_store.cc:276] 0709 12:18:24.033781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.229039 (+2195258us) service_pool.cc:224] Handling call
0709 12:18:26.275218 (+ 46179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.275481 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.275515 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.275588 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:26.275627 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.275699 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.275831 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.294936 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.289538 28118 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:26.303376 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:26.297719 27882 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.284287 28106 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:26.330924 25636 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:26.331104 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.284976 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.331488 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:26.324517 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2079) took 2367 ms. Trace:
I20260709 12:18:26.361913 25708 rpcz_store.cc:276] 0709 12:18:23.956826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.224663 (+2267837us) service_pool.cc:224] Handling call
0709 12:18:26.276563 (+ 51900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.323292 (+ 46729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.324270 (+   978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.332083 28140 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.343561 28133 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:26.345242 27934 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:26.284960 28149 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 12:18:26.367266 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 129134us
I20260709 12:18:26.353425 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.358342 25780 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:26.354527 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:26.182972 27794 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
W20260709 12:18:26.358592 25677 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.032s	sys 0.058s
W20260709 12:18:26.377660 25677 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.033s	sys 0.058s
I20260709 12:18:26.365415 25632 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:26.366376 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 442692us
I20260709 12:18:26.365293 25629 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
W20260709 12:18:26.378086 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 845852us
I20260709 12:18:26.225039 27989 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.295029 28024 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.376430 25638 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:26.285599 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.364084 25634 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:26.380280 25709 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 6. Ops: [4.7-4.7]
I20260709 12:18:26.378196 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 362) took 1982 ms. Trace:
I20260709 12:18:26.388780 25780 rpcz_store.cc:276] 0709 12:18:24.395558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.242794 (+1847236us) service_pool.cc:224] Handling call
0709 12:18:26.357921 (+115127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.376002 (+ 18081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.378135 (+  2133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.393882 27534 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.394655 26344 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:26.398116 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 295) took 2350 ms. Trace:
I20260709 12:18:26.402064 27534 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.408066 26343 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:26.407056 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.410125 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.411032 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 364) took 1956 ms. Trace:
I20260709 12:18:26.411403 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.411473 25781 rpcz_store.cc:276] 0709 12:18:24.454450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.409942 (+1955492us) service_pool.cc:224] Handling call
0709 12:18:26.410350 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.410596 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.410636 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.410716 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.410758 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.410820 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.410943 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.412475 25635 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:26.413410 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.402027 28024 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.415683 26341 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:26.416527 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 296) took 2379 ms. Trace:
I20260709 12:18:26.407406 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.416909 25710 rpcz_store.cc:276] 0709 12:18:24.037437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.221489 (+2184052us) service_pool.cc:224] Handling call
0709 12:18:26.270798 (+ 49309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.416218 (+145420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.416245 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.416284 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.416310 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.416368 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.416479 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.418386 25637 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:26.419000 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.419428 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.419848 27793 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.420161 25637 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:26.420377 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.329702 28108 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.418440 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.421545 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.426856 28106 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:26.427465 28106 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:26.428048 28106 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:26.426940 25710 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:26.429920 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:26.430796 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.433878 25635 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:26.435783 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.436858 25705 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:26.434329 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 492) took 4103 ms. Trace:
I20260709 12:18:26.433197 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:26.443550 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.401461 25709 rpcz_store.cc:276] 0709 12:18:24.037216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.216925 (+2179709us) service_pool.cc:224] Handling call
0709 12:18:26.352836 (+135911us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.387723 (+ 34887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.387996 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:26.445497 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 67784.4us
I20260709 12:18:26.439502 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2084) took 2391 ms. Trace:
I20260709 12:18:26.284087 26340 rpcz_store.cc:276] 0709 12:18:22.603719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.610009 (+  6290us) service_pool.cc:224] Handling call
0709 12:18:22.684404 (+ 74395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.922803 (+2238399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.282964 (+1360161us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54005bf9d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:26.283263 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3196176128}
I20260709 12:18:26.446985 25707 rpcz_store.cc:276] 0709 12:18:24.048417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.363545 (+2315128us) service_pool.cc:224] Handling call
0709 12:18:26.395066 (+ 31521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.439043 (+ 43977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.439088 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.439152 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:26.439201 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.439286 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.439433 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.441314 25631 rpcz_store.cc:276] 0709 12:18:22.330631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.330968 (+   337us) service_pool.cc:224] Handling call
0709 12:18:25.756513 (+3425545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.852709 (+ 96196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.434101 (+581392us) spinlock_profiling.cc:243] Waited 3.42 s on lock 0x7b540018ddd0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:26.434258 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7877162240}
W20260709 12:18:26.454471 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 168463us
I20260709 12:18:26.448472 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2083) took 2402 ms. Trace:
W20260709 12:18:26.455355 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.455673 25713 rpcz_store.cc:276] 0709 12:18:24.046392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.238544 (+2192152us) service_pool.cc:224] Handling call
0709 12:18:26.366861 (+128317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.400935 (+ 34074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.401015 (+    80us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.401148 (+   133us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:26.448209 (+ 47061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.448291 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.448413 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.458381 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2080) took 2499 ms. Trace:
W20260709 12:18:26.459964 25750 connection.cc:256] RPC call timeout handler was delayed by -1.254093168000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
I20260709 12:18:26.458917 25706 rpcz_store.cc:276] 0709 12:18:23.958897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.217040 (+2258143us) service_pool.cc:224] Handling call
0709 12:18:26.354380 (+137340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.457992 (+103612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.458025 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.458074 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:26.458110 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.458178 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.458318 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53342}
I20260709 12:18:26.360692 27936 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.463565 25638 raft_consensus.cc:2410] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:18:26.466903 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.467110 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.468214 25709 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:26.462939 25710 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:26.469125 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.471287 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:26.472216 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:26.473119 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.471508 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.475626 25707 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:26.476217 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.476840 25713 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:26.473781 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:26.480041 25637 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:26.480300 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:26.481340 25706 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:26.481837 25634 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:26.484076 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.486444 27882 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.382156 28148 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:26.511636 28160 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.172s	user 0.004s	sys 0.000s
W20260709 12:18:26.513758 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 59404.5us
I20260709 12:18:26.448060 28141 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.517947 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:26.521410 28139 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.523999 28139 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.525291 28148 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:26.528347 28141 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.472134 27962 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:26.529330 27534 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.528432 28089 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:26.530561 27962 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.530653 28089 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.531096 28089 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:26.535127 28024 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:26.478690 28028 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:18:26.537693 28028 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:26.538018 28028 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:26.447474 28091 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.538870 28089 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.539047 28028 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.540537 28091 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.541348 28089 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.542696 27936 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.542790 27534 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.543609 27534 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:26.544210 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.545524 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.545790 27534 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.546253 25637 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:26.546718 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.546929 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:26.548894 28141 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.549585 28141 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.549980 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:26.550558 26341 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:26.202701 25778 rpcz_store.cc:276] 0709 12:18:23.974413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.974609 (+   196us) service_pool.cc:224] Handling call
0709 12:18:23.974991 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.975378 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.975411 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:23.975928 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.074924 (+ 98996us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:24.312450 (+237526us) log.cc:844] Serialized 14 byte log entry
0709 12:18:24.430291 (+117841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.430432 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.566275 (+135843us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b18000408a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:25.347444 (+781169us) raft_consensus.cc:1685] finished
0709 12:18:25.347580 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.714622 (+367042us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63731,"prepare.queue_time_us":294191,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":104,"raft.queue_time_us":151870,"raft.run_cpu_time_us":697,"raft.run_wall_time_us":27425,"spinlock_wait_cycles":2929024,"thread_start_us":307691,"threads_started":3,"wal-append.queue_time_us":141939}
I20260709 12:18:26.552582 25633 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:26.553548 28133 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.554105 28133 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:26.554867 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1510) took 2575 ms. Trace:
I20260709 12:18:26.556151 28133 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:26.552554 28141 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:26.555220 25784 rpcz_store.cc:276] 0709 12:18:23.979379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.294761 (+315382us) service_pool.cc:224] Handling call
0709 12:18:24.295222 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.798520 (+1503298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.798572 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.799329 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.234082 (+434753us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.234128 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.246995 (+ 12867us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.554577 (+307582us) raft_consensus.cc:1685] finished
0709 12:18:26.554598 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.554780 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80556,"raft.queue_time_us":167117,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":102,"thread_start_us":1382929,"threads_started":2,"wal-append.queue_time_us":662062}
I20260709 12:18:26.558102 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:26.559067 28051 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:26.559659 28051 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.560230 28051 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:26.490831 27794 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.572333 25712 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
W20260709 12:18:26.573192 25712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2014) took 2115 ms (client timeout 2051 ms). Trace:
W20260709 12:18:26.573500 25712 rpcz_store.cc:269] 0709 12:18:24.457450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.767439 (+1309989us) service_pool.cc:224] Handling call
0709 12:18:26.573120 (+805681us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:26.573786 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 104109us
I20260709 12:18:26.576150 28139 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:26.577600 28091 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:26.578162 28091 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:26.578799 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.579394 26341 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:26.579571 25633 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:26.579324 28091 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:26.565271 28065 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
I20260709 12:18:26.582731 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:26.587713 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:26.586536 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 64231.4us
I20260709 12:18:26.573366 28028 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:26.517314 28001 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.589818 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:26.604348 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.596524 28024 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:26.585631 27794 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:26.937618 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:26.798306 27892 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.914s	user 0.008s	sys 0.007s
I20260709 12:18:26.905375 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:26.950605 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.994575 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.902980 25705 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:26.980787 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:26.758981 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2009) took 2481 ms. Trace:
W20260709 12:18:26.981642 27955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.006s	sys 0.010s
I20260709 12:18:26.982383 28065 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.892417 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:26.603070 28062 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.872969 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:26.983332 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:26.992172 25704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.004s	sys 0.013s
I20260709 12:18:26.832043 25710 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:26.879729 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:26.996747 25712 rpcz_store.cc:276] 0709 12:18:24.277225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.606073 (+2328848us) service_pool.cc:224] Handling call
0709 12:18:26.630314 (+ 24241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.723309 (+ 92995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.723350 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.758622 (+ 35272us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.758680 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.758766 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.758910 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.672317 25635 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:26.940429 28044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.005s	sys 0.009s
I20260709 12:18:26.770937 28108 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:26.616286 28139 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:27.000653 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2008) took 2719 ms. Trace:
I20260709 12:18:26.640575 25630 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:26.989542 28001 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:27.003113 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 397861us
I20260709 12:18:26.590240 27952 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:27.004266 25708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.008s	sys 0.004s
I20260709 12:18:27.004142 25784 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:26.826403 28138 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:26.999107 27955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.007s	sys 0.010s
I20260709 12:18:26.996157 27892 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:27.011323 25705 rpcz_store.cc:276] 0709 12:18:24.276845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.484031 (+2207186us) service_pool.cc:224] Handling call
0709 12:18:26.606181 (+122150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.996368 (+390187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.996563 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:27.012090 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:27.014104 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:27.016294 26309 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.012s
W20260709 12:18:27.018384 25708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.008s	sys 0.004s
I20260709 12:18:27.015281 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 363) took 2576 ms. Trace:
I20260709 12:18:27.034322 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:27.007710 28044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.005s	sys 0.012s
W20260709 12:18:27.007490 25704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.005s	sys 0.013s
I20260709 12:18:27.011386 28139 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.022814 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2011) took 2744 ms. Trace:
I20260709 12:18:27.047464 25706 rpcz_store.cc:276] 0709 12:18:24.277886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.021253 (+2743367us) service_pool.cc:224] Handling call
0709 12:18:27.021863 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.022261 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.022303 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.022370 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.022420 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.022537 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.022698 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:27.024457 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:27.036145 26309 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.009s	sys 0.012s
I20260709 12:18:27.036187 25710 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:27.050436 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 596592us
W20260709 12:18:27.033875 28161 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.690s	user 0.011s	sys 0.018s
I20260709 12:18:27.020974 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:27.045675 25787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.008s	sys 0.018s
W20260709 12:18:27.027763 26343 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.007s	sys 0.009s
W20260709 12:18:27.051788 25787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.008s	sys 0.019s
I20260709 12:18:27.041704 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:27.051889 26343 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.007s	sys 0.009s
I20260709 12:18:27.018347 27952 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:27.056759 25779 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:27.007266 28062 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.061939 25710 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:27.067113 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2090) took 1303 ms. Trace:
I20260709 12:18:27.068719 25710 rpcz_store.cc:276] 0709 12:18:25.761542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.050917 (+1289375us) service_pool.cc:224] Handling call
0709 12:18:27.061697 (+ 10780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.064402 (+  2705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.064607 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:27.019376 28065 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.065477 25711 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
W20260709 12:18:27.075531 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.064424 27794 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:27.069810 25751 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.019s	sys 0.026s
W20260709 12:18:27.079713 25751 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.020s	sys 0.026s
W20260709 12:18:27.080080 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 613780us
I20260709 12:18:27.082343 25631 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:27.052260 28139 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:27.079985 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.005s	sys 0.048s
W20260709 12:18:27.088668 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.005s	sys 0.048s
W20260709 12:18:27.089032 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 552325us
I20260709 12:18:27.092325 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.094602 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.095458 26335 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:27.102957 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2091) took 1322 ms. Trace:
I20260709 12:18:27.103350 25709 rpcz_store.cc:276] 0709 12:18:25.761808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.057525 (+1295717us) service_pool.cc:224] Handling call
0709 12:18:27.070003 (+ 12478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.079524 (+  9521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.079551 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.083916 (+  4365us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:27.083976 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.084063 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.084200 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:27.064694 28138 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.107481 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:27.082748 28051 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.522s	user 0.015s	sys 0.000s
W20260709 12:18:27.083029 26344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.028s	sys 0.028s
I20260709 12:18:27.101814 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:27.113359 26344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.028s	sys 0.028s
I20260709 12:18:27.115407 25635 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
W20260709 12:18:27.109623 27534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.019s	sys 0.015s
I20260709 12:18:27.127135 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.131217 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.132537 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:27.130041 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1509) took 3151 ms. Trace:
W20260709 12:18:27.120854 27534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.019s	sys 0.015s
I20260709 12:18:27.084331 27892 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:18:27.123845 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.142681 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.118880 28051 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.134404 25783 rpcz_store.cc:276] 0709 12:18:23.979110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.283823 (+304713us) service_pool.cc:224] Handling call
0709 12:18:24.284314 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.537950 (+1253636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.571465 (+ 33515us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.866916 (+295451us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.211328 (+344412us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:26.211360 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.229194 (+ 17834us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.026044 (+796850us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b2400243de8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27289c4 00007f29b2726acb 00007f29b28e1dc9 00007f29b28e1a55 00007f29b28e109d 00007f29b551310e 00007f29b552a88c 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221
0709 12:18:27.111799 (+ 85755us) raft_consensus.cc:1685] finished
0709 12:18:27.111825 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.129969 (+ 18144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75978,"raft.queue_time_us":162648,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"spinlock_wait_cycles":81506048,"thread_start_us":1241991,"threads_started":2,"wal-append.queue_time_us":443146}
I20260709 12:18:27.157595 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.158257 26336 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
W20260709 12:18:27.159637 25751 leader_election.cc:336] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.133s (SENT)
I20260709 12:18:27.160265 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:27.161319 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:27.163307 28168 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:18:27.164240 28167 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:18:27.164935 26340 raft_consensus.cc:2368] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4.
I20260709 12:18:27.167425 25779 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:27.166364 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:27.170564 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.169420 28140 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:27.175907 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.176462 28142 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:27.176612 26340 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:27.176959 28142 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:27.179090 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.177783 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.174108 26307 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:27.195030 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:27.196455 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:27.197275 28089 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:27.197364 28044 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:27.202512 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.200191 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.201711 28044 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.205322 28044 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.207248 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.201678 27955 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:27.227015 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53067.7us
I20260709 12:18:27.228494 28024 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:27.228914 28024 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.229121 27955 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:27.229300 28024 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.229607 27955 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.151089 27952 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.233558 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.233899 27952 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:27.234315 25781 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:27.235512 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.236438 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.237825 25783 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:27.239140 28173 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:18:27.180858 28175 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:18:27.202142 28089 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:27.282908 25751 leader_election.cc:336] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.512s (SENT)
I20260709 12:18:27.283560 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:27.288429 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:27.319769 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.203585 26340 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:27.238235 25781 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:26.880681 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:27.295480 28106 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.858s	user 0.005s	sys 0.008s
I20260709 12:18:27.177999 28142 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:27.286589 28089 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:27.320724 26336 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:27.282944 28108 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.331439 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.327055 25779 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:27.329717 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 361) took 2949 ms. Trace:
I20260709 12:18:27.280918 28065 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:27.349404 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 121482us
I20260709 12:18:27.383360 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2077) took 3429 ms. Trace:
I20260709 12:18:27.383932 25704 rpcz_store.cc:276] 0709 12:18:23.954245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.760841 (+1806596us) service_pool.cc:224] Handling call
0709 12:18:25.761255 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.761714 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:26.227966 (+466252us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.382702 (+1154736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.382774 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:27.382833 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.382900 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.383302 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8072,"prepare.queue_time_us":636733,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":122996,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":27236,"thread_start_us":1447382,"threads_started":3,"wal-append.queue_time_us":831729}
I20260709 12:18:27.384765 25532 catalog_manager.cc:5697] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:18:27.330593 26335 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:27.387092 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2088) took 3010 ms. Trace:
I20260709 12:18:27.387548 25706 rpcz_store.cc:276] 0709 12:18:24.376247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.048638 (+2672391us) service_pool.cc:224] Handling call
0709 12:18:27.049048 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.049326 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:27.049679 (+   353us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.386699 (+337020us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.386747 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:27.386859 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.386927 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.387033 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4280,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"thread_start_us":336498,"threads_started":1,"wal-append.queue_time_us":297762}
I20260709 12:18:27.389212 25532 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:27.380416 25785 rpcz_store.cc:276] 0709 12:18:24.380495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.240181 (+1859686us) service_pool.cc:224] Handling call
0709 12:18:26.275492 (+ 35311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.512415 (+236923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.512448 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.518899 (+  6451us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.518939 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.329489 (+810550us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400191254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:27.329537 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.329655 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":748,"raft.queue_time_us":1897,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8033,"spinlock_wait_cycles":476846848}
W20260709 12:18:27.382320 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 339708us
W20260709 12:18:27.378525 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 260053us
I20260709 12:18:27.390815 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:27.395056 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:27.393576 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:27.409713 25707 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:27.379468 28175 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:27.398058 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:27.442909 28175 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:27.036541 25786 rpcz_store.cc:276] 0709 12:18:24.439511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.250703 (+1811192us) service_pool.cc:224] Handling call
0709 12:18:26.362405 (+111702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.472592 (+110187us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:26.567061 (+ 94469us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.014882 (+447821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.014957 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:27.015015 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.015088 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.015208 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":486,"prepare.queue_time_us":82204,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":7748,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":12093,"thread_start_us":392471,"threads_started":1}
I20260709 12:18:27.443996 28175 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.444789 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:27.445377 28175 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:27.446146 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:27.449127 25713 raft_consensus.cc:2376] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 3. Re-sending same reply.
I20260709 12:18:27.417405 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.391249 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.451817 26337 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.452350 27989 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:27.454869 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:27.455826 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.456838 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:27.418709 28142 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:27.419090 25711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2081) took 3416 ms (client timeout 2111 ms). Trace:
W20260709 12:18:27.460131 25711 rpcz_store.cc:269] 0709 12:18:24.002364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.717363 (+1714999us) service_pool.cc:224] Handling call
0709 12:18:27.418876 (+1701513us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54003c7dd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:27.419002 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1986650880}
I20260709 12:18:27.461434 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:27.462070 25711 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:27.422315 27987 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:27.413033 28166 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:18:27.390159 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:27.463001 25713 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:27.463836 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 357) took 3143 ms. Trace:
W20260709 12:18:27.430984 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 165048us
I20260709 12:18:27.436005 28051 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:27.440725 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1549) took 1253 ms. Trace:
I20260709 12:18:27.464474 25787 rpcz_store.cc:276] 0709 12:18:24.320715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.824523 (+1503808us) service_pool.cc:224] Handling call
0709 12:18:25.908164 (+ 83641us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:26.147564 (+239400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.147606 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:26.236194 (+ 88588us) log.cc:844] Serialized 52 byte log entry
0709 12:18:27.052250 (+816056us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:27.052301 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.052389 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.463629 (+411240us) raft_consensus.cc:1685] finished
0709 12:18:27.463647 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.463779 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18425,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":7916,"thread_start_us":713384,"threads_started":1,"wal-append.queue_time_us":850913}
I20260709 12:18:27.465693 25630 rpcz_store.cc:276] 0709 12:18:26.187483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.289943 (+102460us) service_pool.cc:224] Handling call
0709 12:18:27.440490 (+1150547us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54002a5b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:27.440653 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654494464}
I20260709 12:18:27.416342 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:27.466893 25637 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.375056 28140 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:27.471288 25713 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:27.459441 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.463763 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:27.418098 28178 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 12:18:27.475665 25706 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:18:27.476080 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51360us
W20260709 12:18:27.476682 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 74353us
I20260709 12:18:27.435985 28138 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:27.448561 28062 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:27.473067 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78113.1us
I20260709 12:18:27.417279 25633 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:27.472002 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.479763 25783 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:27.485890 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2099) took 1608 ms. Trace:
I20260709 12:18:27.487461 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:27.487763 25713 rpcz_store.cc:276] 0709 12:18:25.865951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.470762 (+1604811us) service_pool.cc:224] Handling call
0709 12:18:27.471083 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.473681 (+  2598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.473855 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:27.339468 28106 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.498831 25534 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 192) took 11112 ms. Trace:
I20260709 12:18:27.502383 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.505188 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:27.500243 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:27.507308 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.487789 27952 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:27.510576 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.528148 25785 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:27.528036 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.515496 25637 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
W20260709 12:18:27.522891 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.508414 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.503077 25787 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:27.530882 25712 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:27.411098 28108 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.535840 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:27.544628 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44498.4us
I20260709 12:18:27.546624 25709 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:27.547407 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.548123 25787 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:27.502640 28138 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:27.551512 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 62751us
W20260709 12:18:27.552228 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53212.8us
I20260709 12:18:27.472187 28044 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:27.555241 26338 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:18:27.590276 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.550004 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.527508 27952 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:27.499294 25534 rpcz_store.cc:276] 0709 12:18:16.386564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.387016 (+   452us) service_pool.cc:224] Handling call
0709 12:18:27.498523 (+11111507us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:19.116700 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:19.371786 (+255086us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:19.371828 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:18:19.371846 (+    18us) tablet.cc:662] Decoding operations
0709 12:18:19.372739 (+   893us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:19.372796 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:19.372826 (+    30us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:18:19.373435 (+   609us) tablet.cc:701] Row locks acquired
0709 12:18:19.373454 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:18:19.373625 (+   171us) write_op.cc:270] Start()
0709 12:18:19.373743 (+   118us) write_op.cc:276] Timestamp: P: 1783599499373563 usec, L: 0
0709 12:18:19.373767 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:18:19.374428 (+   661us) log.cc:844] Serialized 2835 byte log entry
0709 12:18:20.233414 (+858986us) op_driver.cc:464] REPLICATION: finished
0709 12:18:20.680159 (+446745us) write_op.cc:301] APPLY: starting
0709 12:18:20.688389 (+  8230us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:20.721841 (+ 33452us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:20.762196 (+ 40355us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:20.884614 (+122418us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:20.884868 (+   254us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:18:20.884913 (+    45us) write_op.cc:312] APPLY: finished
0709 12:18:20.885170 (+   257us) log.cc:844] Serialized 59 byte log entry
0709 12:18:20.885341 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:20.885460 (+   119us) write_op.cc:454] Released schema lock
0709 12:18:20.885514 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:20.885897 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:20.885926 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":63776,"reported_tablets":20,"spinlock_wait_cycles":7424,"child_traces":[["op",{"apply.queue_time_us":393346,"apply.run_cpu_time_us":3768,"apply.run_wall_time_us":235775,"num_ops":7,"prepare.queue_time_us":188185,"prepare.run_cpu_time_us":42967,"prepare.run_wall_time_us":559866,"raft.queue_time_us":321495,"raft.run_cpu_time_us":13069,"raft.run_wall_time_us":341462,"replication_time_us":859458,"thread_start_us":827352,"threads_started":4,"wal-append.queue_time_us":449106,"wal-append.run_cpu_time_us":23835,"wal-append.run_wall_time_us":2034188}]]}
I20260709 12:18:27.558512 28186 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:27.571980 28106 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:27.576004 28108 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:27.582599 26339 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:18:27.592813 25629 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:27.593441 28139 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:27.584775 28024 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:27.587229 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.588171 27936 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:18:27.590370 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.540643 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.495958 27794 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:18:27.602349 25707 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:27.596606 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:27.619534 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 235352us
W20260709 12:18:27.597990 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 62253.5us
I20260709 12:18:27.604902 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2097) took 1740 ms. Trace:
I20260709 12:18:27.623844 25709 rpcz_store.cc:276] 0709 12:18:25.864921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.104276 (+1239355us) service_pool.cc:224] Handling call
0709 12:18:27.542654 (+438378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.554245 (+ 11591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.604694 (+ 50449us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400236250. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:27.604838 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979271424}
I20260709 12:18:27.534788 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:27.628312 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2016) took 1224 ms. Trace:
I20260709 12:18:27.628819 25706 rpcz_store.cc:276] 0709 12:18:26.403655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.509979 (+1106324us) service_pool.cc:224] Handling call
0709 12:18:27.544224 (+ 34245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.614814 (+ 70590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.614861 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.627897 (+ 13036us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:27.627955 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.628078 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.628230 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9972}
I20260709 12:18:27.610983 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.540433 28140 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:27.612342 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 136461us
I20260709 12:18:27.648545 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.649295 25781 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:27.591183 27952 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.650314 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.651093 25786 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:27.652173 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.652787 25780 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:27.653192 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.653898 25785 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:27.655452 25783 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:27.659654 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.661396 25781 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:18:27.662156 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.650377 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:27.663627 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60465.9us
I20260709 12:18:27.664067 25713 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:18:27.666868 28178 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:18:27.667833 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 358) took 3346 ms. Trace:
I20260709 12:18:27.669140 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.668221 25782 rpcz_store.cc:276] 0709 12:18:24.321041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.861998 (+1540957us) service_pool.cc:224] Handling call
0709 12:18:25.886964 (+ 24966us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:26.474312 (+587348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.474344 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:26.513800 (+ 39456us) log.cc:844] Serialized 52 byte log entry
0709 12:18:27.019488 (+505688us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:27.051341 (+ 31853us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.051670 (+   329us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.051815 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.051898 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.231881 (+179983us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b180040df20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:27.239109 (+  7228us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b4800250158. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29ba8b8aa1 00007f29ba8afb37 00007f29ba8af9ca 00007f29ba9a726f 00007f29ba9a6f37 00007f29ba9bb55c 00007f29ba9bb508 00007f29ba9bb488 00007f29ba9bb443 00007f29ba9ba6ec 00007f29b98e4a96
0709 12:18:27.456337 (+217228us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b180040df20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:27.667599 (+211262us) raft_consensus.cc:1685] finished
0709 12:18:27.667622 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.667773 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121256,"prepare.queue_time_us":213016,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":104939,"raft.run_cpu_time_us":1512,"raft.run_wall_time_us":195691,"spinlock_wait_cycles":26726400,"thread_start_us":428544,"threads_started":1,"wal-append.queue_time_us":584036}
I20260709 12:18:27.546324 28051 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:18:27.670473 27955 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:27.672277 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 101875us
I20260709 12:18:27.672803 27955 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:27.673482 27955 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:27.676338 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2085) took 3616 ms. Trace:
I20260709 12:18:27.616616 25712 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:27.558138 26337 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:27.678040 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2027) took 1214 ms. Trace:
I20260709 12:18:27.676844 25708 rpcz_store.cc:276] 0709 12:18:24.060315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.365731 (+2305416us) service_pool.cc:224] Handling call
0709 12:18:26.446674 (+ 80943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.469989 (+ 23315us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:27.040172 (+570183us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.604085 (+563913us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.618361 (+ 14276us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.619466 (+  1105us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.619517 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.619592 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.676117 (+ 56525us) raft_consensus.cc:1685] finished
0709 12:18:27.676137 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.676274 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51647,"prepare.queue_time_us":16280,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":36,"raft.queue_time_us":51975,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":15163,"thread_start_us":986339,"threads_started":2,"wal-append.queue_time_us":559262}
I20260709 12:18:27.678395 25712 rpcz_store.cc:276] 0709 12:18:26.464041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.597475 (+1133434us) service_pool.cc:224] Handling call
0709 12:18:27.616202 (+ 18727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.677777 (+ 61575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.677979 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:27.680290 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:27.682279 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59241.2us
I20260709 12:18:27.682425 25633 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:27.684867 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:27.690014 25709 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:27.692569 28089 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:27.695667 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.696280 25778 raft_consensus.cc:1217] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Deduplicated request from leader. Original: 1.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:18:27.696509 26342 raft_consensus.cc:2368] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:27.696863 25778 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:18:27.674378 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:27.702099 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:27.633827 26336 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 3 LEARNER]: Advancing to term 5
I20260709 12:18:27.698333 25707 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:18:27.553634 28138 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:27.725069 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.717758 25787 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
W20260709 12:18:27.719156 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 100451us
I20260709 12:18:27.676326 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2012) took 3398 ms. Trace:
I20260709 12:18:27.536701 28188 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.179s
I20260709 12:18:27.731642 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:27.733095 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2098) took 1867 ms. Trace:
I20260709 12:18:27.768079 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:27.772732 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 122570us
W20260709 12:18:27.785766 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 100988us
I20260709 12:18:27.786182 25633 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:27.775667 25677 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:27.726099 25705 rpcz_store.cc:276] 0709 12:18:24.278212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.042977 (+2764765us) service_pool.cc:224] Handling call
0709 12:18:27.072996 (+ 30019us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.102170 (+ 29174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.102197 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.115734 (+ 13537us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.538066 (+422332us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:27.538123 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.538212 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.675986 (+137774us) raft_consensus.cc:1685] finished
0709 12:18:27.676042 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.676240 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112159,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":386895,"threads_started":1,"wal-append.queue_time_us":535020}
W20260709 12:18:27.795696 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 175136us
I20260709 12:18:27.731467 27955 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:27.731789 27534 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:27.798058 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.770273 25711 rpcz_store.cc:276] 0709 12:18:25.865661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.469868 (+1604207us) service_pool.cc:224] Handling call
0709 12:18:27.470139 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.470387 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:27.471692 (+  1305us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.732653 (+260961us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.732711 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:27.732817 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.732908 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.733030 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":905,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":250545,"threads_started":1}
I20260709 12:18:27.723349 28138 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:27.720600 25704 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:27.804507 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72029.9us
I20260709 12:18:27.829100 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.829375 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:27.772444 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.807070 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.834517 25781 raft_consensus.cc:2410] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:27.808198 25782 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:18:27.808795 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 205432us
I20260709 12:18:27.797254 28051 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:27.811748 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.812629 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:27.829866 25787 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:27.832484 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.809154 25706 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:18:27.850963 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 118823us
I20260709 12:18:27.841298 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:27.854391 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:27.855840 25637 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:27.806946 27923 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:27.827677 26337 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:27.859500 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.813465 28191 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.864372 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.733361 28044 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:27.867904 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:27.875290 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:27.833317 28089 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:27.846242 28001 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:27.873413 25603 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:27.880766 28051 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:27.853578 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.884414 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.885802 28044 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:27.865638 27892 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:27.860576 27923 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:27.864130 27936 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:27.951574 28180 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:27.952031 27936 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:27.952725 25637 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:18:27.954165 28182 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:27.954650 28182 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:27.954877 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:27.955495 28182 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:27.956887 28182 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.959278 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:27.961242 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.942706 28194 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:18:27.962368 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:27.962980 26344 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.871884 28091 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:27.879343 28065 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:27.946913 28201 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:27.947970 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:27.948860 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:27.949707 28108 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:27.882973 28001 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.887707 28191 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:27.875757 28140 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:27.939043 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:27.973044 28028 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.360s	user 0.013s	sys 0.001s
I20260709 12:18:28.005326 26341 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:27.986831 28065 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.006196 28065 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:27.975988 27923 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:28.007496 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2093) took 2245 ms. Trace:
I20260709 12:18:27.970654 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:28.008092 25710 rpcz_store.cc:276] 0709 12:18:25.762373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.069449 (+1307076us) service_pool.cc:224] Handling call
0709 12:18:27.104743 (+ 35294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.397817 (+293074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.397878 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.427011 (+ 29133us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.619197 (+192186us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.619248 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.619338 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.007155 (+387817us) raft_consensus.cc:1685] finished
0709 12:18:28.007188 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.007404 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76129,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":464178,"threads_started":2,"wal-append.queue_time_us":439952}
I20260709 12:18:28.009470 28142 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:28.012748 28142 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.012833 28196 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.013211 28142 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.013595 28024 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:28.013461 28196 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.014884 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.015422 25705 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:27.883404 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:28.016472 26340 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:28.018860 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.019564 25779 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:28.019695 26344 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:28.022665 28142 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.005721 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:27.987046 25785 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:27.988390 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 119166us
I20260709 12:18:27.992908 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.024094 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.025197 25711 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:28.025835 26342 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:27.974673 26336 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 5 LEARNER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
W20260709 12:18:28.027345 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 139827us
W20260709 12:18:28.026218 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 150996us
I20260709 12:18:27.969293 28139 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:28.023842 25631 raft_consensus.cc:2376] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4. Re-sending same reply.
I20260709 12:18:27.952028 28180 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:28.028861 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:28.028949 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 164820us
I20260709 12:18:27.987128 28192 raft_consensus.cc:2764] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:28.033026 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.033356 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:28.034013 25785 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.034754 28175 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:28.035219 28175 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.035583 28175 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.036577 25704 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:28.037864 28180 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.033031 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.011319 28001 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.011047 26338 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:28.030467 28106 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:28.040032 28106 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:28.040482 28106 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:27.979911 28186 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:18:28.011014 28140 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.042920 28140 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:28.043309 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.043609 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.044122 25787 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:28.044227 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.044767 25781 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:28.044873 25780 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:27.975274 28089 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:28.046169 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.047595 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.048957 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.034109 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.050557 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:28.051017 28188 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:28.051206 25780 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.051499 28188 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:28.052013 28188 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.052738 28089 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.053726 25677 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.054000 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:28.030761 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:28.056833 25785 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:28.061173 25705 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:28.061563 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:27.939414 28044 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:28.075575 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46895.4us
I20260709 12:18:28.039215 28180 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.062863 27989 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:28.083165 27989 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:28.083571 27989 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:28.084033 26341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 672) took 1451 ms (client timeout 1648 ms). Trace:
W20260709 12:18:28.084429 26341 rpcz_store.cc:269] 0709 12:18:26.632188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.982228 (+350040us) service_pool.cc:224] Handling call
0709 12:18:28.083831 (+1101603us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:28.083936 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2322814848}
I20260709 12:18:28.072592 28192 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:18:28.086074 28192 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:28.086409 28192 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:28.087320 28192 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.088505 28192 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.064697 28044 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:28.050182 28106 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.098340 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:28.065218 28062 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:28.010295 27952 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.068624 28196 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:28.066570 28191 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:28.065343 27955 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:18:28.072907 28175 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.092355 27987 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:18:28.119448 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:28.122673 28089 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:28.138618 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 84964.2us
I20260709 12:18:28.159252 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.128782 28065 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.132862 28001 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.148664 26338 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:28.164731 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.165719 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.119284 28051 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.167721 25787 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:28.162268 28089 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:28.169631 28065 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:28.179878 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.173095 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.181073 25781 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:28.181607 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 172103us
W20260709 12:18:28.180949 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 109348us
W20260709 12:18:28.183547 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 77309.2us
W20260709 12:18:28.189913 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 113066us
I20260709 12:18:28.165475 28062 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:28.195266 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.174239 28089 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.179390 28142 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:28.231338 28106 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:28.231734 28001 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:28.237839 28001 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:28.237927 28062 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.237962 28175 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:28.247668 28175 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:28.248292 28175 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.251641 28106 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:28.252118 28106 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.236610 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.257438 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.160225 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:28.252486 28106 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:28.263236 25785 raft_consensus.cc:606] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:28.265651 25785 raft_consensus.cc:614] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:28.201354 28089 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.233544 28065 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:18:28.202692 28202 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:18:28.272927 28187 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:18:28.184304 27936 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:18:28.200166 27987 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:28.200248 28108 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:18:28.282147 27923 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.287206 25706 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:28.262858 28201 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:28.272049 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:28.296929 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.267117 28175 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:28.300578 27989 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:28.304447 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.307188 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:28.310348 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 123054us
I20260709 12:18:28.308504 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:28.277992 28191 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:28.278651 28065 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:28.314850 25786 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.331429 25631 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:28.198546 28204 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:18:28.320230 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:28.339191 25635 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.181098 28213 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 12:18:28.337280 25629 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:28.319536 28201 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.314567 27952 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:28.352710 27923 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:28.353336 28191 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:28.354357 28191 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.358621 28222 raft_consensus.cc:1064] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:18:28.361321 25603 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.363230 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.364079 25712 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.364382 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.365144 25710 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.365993 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.366194 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.366716 25708 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:28.366855 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.367516 25704 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.370028 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:28.370707 25785 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:28.371515 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:28.374550 27987 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:28.375068 27987 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:28.375964 27987 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.378615 28091 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:28.379091 28091 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:28.384274 25778 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:28.386008 25780 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:28.387718 27989 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:28.390283 28217 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:18:28.287257 28219 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:18:28.366063 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.424561 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:28.436628 25710 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.437683 28142 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:28.438230 28142 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.438601 28142 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:28.440173 28196 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:28.441408 28196 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:28.441843 28196 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:28.443111 28196 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.444700 28196 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:28.452149 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 423973us
W20260709 12:18:28.454994 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 61210.5us
I20260709 12:18:28.456211 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:28.312536 28140 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.470927 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.471565 25706 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.349593 28201 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:28.467367 25676 connection.cc:256] RPC call timeout handler was delayed by -2.6241447059999246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 12:18:28.472627 25676 connection.cc:256] RPC call timeout handler was delayed by -2.1237447059997976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 12:18:28.473107 25676 connection.cc:256] RPC call timeout handler was delayed by -1.9452332690000276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 12:18:28.473454 25676 connection.cc:256] RPC call timeout handler was delayed by -1.7238332690001243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
W20260709 12:18:28.473934 25676 connection.cc:256] RPC call timeout handler was delayed by -1.6608332690000225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
W20260709 12:18:28.474411 25676 connection.cc:256] RPC call timeout handler was delayed by -1.4745332689999486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2391s for a response
I20260709 12:18:28.348433 28213 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:28.475766 25710 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:28.469197 28180 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.477648 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.479071 25708 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:28.481969 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55827.5us
I20260709 12:18:28.482757 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.483559 25782 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:28.432619 28044 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.488227 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.511238 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.513275 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.514144 25708 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.514997 25635 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:28.515302 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.520301 25710 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:28.531049 25704 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
W20260709 12:18:28.488881 28138 consensus_meta.cc:287] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.641s	user 0.008s	sys 0.003s
I20260709 12:18:28.539062 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.464903 28220 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:18:28.474835 28028 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 12:18:28.543174 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 65859.5us
I20260709 12:18:28.551131 25704 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:28.523392 28139 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:28.566563 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.576637 28028 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:28.583218 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 103431us
W20260709 12:18:28.584981 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.521183 28180 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.588531 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.544143 28024 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:18:28.580350 28028 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:28.590107 25704 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.568154 25778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a760df354a14284b004635d45d88a3e"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:28.575591 28220 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:28.573236 28139 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:28.546994 28138 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:28.598690 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.608954 25710 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:28.518847 28065 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:28.603507 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 85666.3us
I20260709 12:18:28.591344 27989 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:28.605989 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:28.615340 28230 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:18:28.574203 27936 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:18:28.622159 25706 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:28.630508 28226 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:28.613057 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:28.609385 28139 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.605516 28220 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:28.635687 25779 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:28.611516 25778 raft_consensus.cc:606] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:28.632202 28192 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.006s
I20260709 12:18:28.650067 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.650868 25706 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.654616 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.635178 28065 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:28.655318 25704 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:28.658929 28024 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:28.659730 28024 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:28.659886 28044 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:28.660239 28227 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:18:28.660837 28240 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:28.563583 28222 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.657747 28140 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:28.665571 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 148465us
I20260709 12:18:28.666066 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.666797 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:28.658353 28188 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.000s
W20260709 12:18:28.674196 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 218066us
W20260709 12:18:28.671026 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 491234us
I20260709 12:18:28.678304 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.678673 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.679510 25704 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 2.
I20260709 12:18:28.630861 28028 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.672537 28024 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.687968 25603 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.691586 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:28.692097 25601 leader_election.cc:400] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4.
I20260709 12:18:28.695616 25601 leader_election.cc:403] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:18:28.694806 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 81816.7us
I20260709 12:18:28.705073 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:28.706429 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:28.692373 28236 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:18:28.710986 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:28.711268 28241 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:18:28.712378 25710 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:28.713167 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.714113 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.714811 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.715063 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.715519 25709 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:28.715791 25710 raft_consensus.cc:2410] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:28.716274 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.716491 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.717125 25710 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.717379 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.717810 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.718044 25713 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.718689 28234 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:28.718849 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.690593 28218 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:28.719993 28138 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:28.723151 28232 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:18:28.723222 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:28.720418 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:28.724901 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.725230 25786 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:28.720192 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.718456 25710 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.726821 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:28.693120 27952 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:28.727093 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.727186 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.719413 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.722841 28182 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:28.726156 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:28.730852 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.725620 25705 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:28.682838 28220 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.675375 28188 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.808706 25633 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
W20260709 12:18:28.862717 26309 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.792s (SENT) [suppressed 50 similar messages]
I20260709 12:18:28.885601 25704 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 3: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:28.690603 28229 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:28.891112 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.892385 25709 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:28.763049 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 72804.1us
I20260709 12:18:28.764235 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.808224 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:28.903980 25706 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:28.736791 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:28.829826 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:28.886935 26341 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.728951 25713 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:28.727568 26344 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:28.758806 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:28.883469 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 67908.4us
I20260709 12:18:28.762631 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:28.796509 25751 leader_election.cc:336] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.622s (SENT)
W20260709 12:18:28.816707 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 125364us
W20260709 12:18:28.862125 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.907758 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:28.704726 26308 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:28.909452 25635 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:28.763850 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:28.911510 26343 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:28.909660 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.920277 25709 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:28.722260 28218 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.722666 28246 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:28.764194 28219 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:28.814882 27534 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:28.818631 28028 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:28.931874 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:28.816051 28024 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:28.761174 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:28.760263 28213 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.728297 27952 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:28.949863 26309 leader_election.cc:336] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.567s (SENT)
I20260709 12:18:28.763476 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:28.816362 28091 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.802768 28180 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:28.766364 28234 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:28.903769 28229 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.710273 28142 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:28.890434 28182 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:28.952386 27534 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:28.976580 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 180149us
W20260709 12:18:28.975154 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 47576.8us
I20260709 12:18:29.002831 26338 raft_consensus.cc:2376] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 5. Re-sending same reply.
I20260709 12:18:29.007427 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:28.922708 27934 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:29.013062 26341 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
W20260709 12:18:28.954017 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 217868us
I20260709 12:18:28.805778 28201 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:28.956015 28089 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.670s	user 0.010s	sys 0.001s
W20260709 12:18:29.017607 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 312934us
I20260709 12:18:28.946762 28218 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:29.036082 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.948362 28138 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.976559 25712 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:28.982316 28204 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:28.981192 28219 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:29.069365 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:29.040884 28229 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:29.095598 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.090009 28106 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.802s	user 0.011s	sys 0.003s
W20260709 12:18:29.106068 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 156218us
W20260709 12:18:29.108546 26309 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.546s (SENT)
I20260709 12:18:29.101132 28191 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:29.103696 28196 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:29.126893 26309 leader_election.cc:336] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.555s (SENT)
I20260709 12:18:29.127586 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:29.129904 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:29.130769 25633 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:29.092161 28138 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.101040 27934 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:18:29.098078 28089 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:29.156855 25712 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:29.155344 28268 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:29.163661 28257 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:18:29.121204 28182 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.171495 28182 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:29.172150 28244 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.376s	user 0.002s	sys 0.000s
W20260709 12:18:29.174916 28254 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:18:29.112993 28106 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.178328 25709 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:29.179692 25704 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:29.181248 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 273023us
I20260709 12:18:28.763358 28253 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4.
I20260709 12:18:29.133127 28191 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:29.188196 25707 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:29.192006 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 65162.3us
I20260709 12:18:29.110930 27534 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.194424 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:29.206055 25711 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:18:29.209332 26343 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
W20260709 12:18:29.209681 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 116938us
I20260709 12:18:29.177977 28188 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:28.997646 28246 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:29.214540 25635 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:29.216174 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:29.189733 27934 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:29.015964 28213 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.223553 25706 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:18:29.233996 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.242803 25637 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:29.143019 28219 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:29.245993 25709 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:29.225476 27534 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:29.243891 25712 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:29.113963 28234 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.221312 28091 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:29.249086 28208 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:29.249667 28251 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 12:18:29.250133 28259 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:18:29.228612 28142 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:29.254231 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2044) took 1077 ms. Trace:
W20260709 12:18:29.268576 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 194566us
I20260709 12:18:29.266386 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:29.273191 25707 rpcz_store.cc:276] 0709 12:18:28.176872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.184123 (+  7251us) service_pool.cc:224] Handling call
0709 12:18:29.253923 (+1069800us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b5400204254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:29.254132 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280851200}
W20260709 12:18:29.280121 26309 leader_election.cc:336] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.721s (SENT)
I20260709 12:18:29.284521 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:29.285208 25751 leader_election.cc:336] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.648s (SENT)
I20260709 12:18:29.285840 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:29.285987 26309 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.767s (SENT)
I20260709 12:18:29.291783 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:29.293586 26309 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.804s (SENT)
W20260709 12:18:29.299026 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 51664.7us
W20260709 12:18:29.299455 25751 leader_election.cc:336] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.723s (SENT)
I20260709 12:18:29.299885 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:29.300109 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:29.300527 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 412553us
W20260709 12:18:29.306107 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.276001 28220 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:29.312888 25633 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:29.270234 28264 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:29.276386 28201 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:29.290833 28091 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:29.323241 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:29.326126 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:29.295560 28269 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:29.304908 28204 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:29.355580 26342 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:29.308807 28266 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:29.341956 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.832s (SENT)
I20260709 12:18:29.343986 25633 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:29.344761 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.346016 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:29.354666 26309 leader_election.cc:336] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.810s (SENT)
I20260709 12:18:29.341061 28258 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:29.383219 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:29.383504 25782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:29.383957 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:29.384436 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:29.319984 27955 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:29.387697 26339 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:18:29.386446 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 72401.4us
I20260709 12:18:29.394203 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:29.276690 28191 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.356667 28252 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:29.393988 28139 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.000s
I20260709 12:18:29.436184 25712 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:29.385998 28208 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:29.440215 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:29.445503 25713 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:29.446455 26338 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:18:29.383025 28267 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:29.411913 28138 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:29.467696 25629 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:29.424961 28253 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:29.449085 28028 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.471678 28273 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:29.472555 28278 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:29.431924 28106 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:29.383602 28269 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.490770 28089 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:29.495211 28272 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:18:29.497071 28246 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:29.465401 28282 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:29.488682 27952 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:29.524528 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:29.562379 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.569334 25629 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:29.481994 28028 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:29.601130 28028 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:29.596032 25713 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:29.608949 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.617694 25713 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:29.618512 25709 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:29.629788 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:29.630457 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 291593us
I20260709 12:18:29.640375 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:29.630832 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.642225 28258 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:29.444352 28139 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.476759 28267 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:29.634003 25708 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:29.630908 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 288978us
W20260709 12:18:29.653492 25751 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.921s (SENT)
W20260709 12:18:29.606410 28277 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:18:29.646160 25712 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:18:29.655196 25631 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.566s	user 0.014s	sys 0.003s
W20260709 12:18:29.653237 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 322817us
I20260709 12:18:29.653844 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:29.608952 28246 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:29.655783 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.654217 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:29.655628 25631 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:29.606865 28253 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:29.663412 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:29.664152 25636 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 2.
I20260709 12:18:29.666281 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:29.643375 28264 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:29.640091 27952 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:29.674068 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:29.686913 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:29.688663 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:29.725894 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.797367 28258 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:29.674943 28219 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.664604 27955 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:18:29.797372 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 143882us
I20260709 12:18:29.861743 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:29.718374 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:29.904737 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.054s	sys 0.102s
I20260709 12:18:29.675668 28274 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:29.811693 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.104524 25638 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:29.691427 28210 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:29.820840 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.105476 25634 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:29.674755 28089 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:29.668587 28246 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:29.659813 28106 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.189s
I20260709 12:18:29.810886 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:29.663848 28091 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.664986 28260 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:29.848133 28051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.014s	sys 0.032s
I20260709 12:18:29.960855 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:29.958691 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.963284 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.068850 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:29.965688 25712 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:30.069640 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:29.955264 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:30.076479 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.067s	sys 0.111s
I20260709 12:18:30.095330 25636 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.095888 25637 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:18:29.969708 28280 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.387s	user 0.001s	sys 0.000s
W20260709 12:18:29.918444 28268 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.034s	sys 0.024s
W20260709 12:18:30.099572 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.960608 28213 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:30.126931 25632 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.128875 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:30.152388 25709 raft_consensus.cc:2368] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7.
W20260709 12:18:30.136580 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 181387us
I20260709 12:18:30.137192 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.094405 28028 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.144098 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:30.119371 28245 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.323s	user 0.000s	sys 0.018s
I20260709 12:18:30.126660 28188 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.154695 25751 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.794s (SENT)
I20260709 12:18:30.171542 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:30.157519 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.029528 28219 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:30.144418 28182 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.972s	user 0.015s	sys 0.009s
I20260709 12:18:30.161933 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:30.120985 28089 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:30.178210 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:30.141160 28051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.021s	sys 0.039s
I20260709 12:18:30.176684 25637 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:30.187853 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.188445 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.188843 25637 raft_consensus.cc:2376] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 5. Re-sending same reply.
I20260709 12:18:30.160777 25630 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.128715 28260 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:30.183413 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:30.185596 26309 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.027s	sys 0.037s
I20260709 12:18:30.067714 28264 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.195376 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:30.173349 28263 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.588s	user 0.000s	sys 0.001s
I20260709 12:18:30.144619 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:30.197757 25787 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:30.158584 28106 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:30.198952 25630 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:30.197767 26309 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.027s	sys 0.037s
I20260709 12:18:30.145637 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.196626 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.208032 25786 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:18:30.140479 28091 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.174401 28213 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:18:30.233210 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 576606us
I20260709 12:18:30.186600 28089 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:30.271731 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.235441 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.144490 28285 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:30.236630 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.193018 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:30.243824 25629 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.029s	sys 0.069s
W20260709 12:18:30.258529 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:30.147282 28268 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.035s	sys 0.025s
W20260709 12:18:30.149082 28024 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.169s	user 0.008s	sys 0.015s
I20260709 12:18:30.091324 28267 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:30.359591 25629 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.029s	sys 0.085s
W20260709 12:18:30.372269 25631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1562) took 1675 ms (client timeout 1693 ms). Trace:
W20260709 12:18:30.372926 25631 rpcz_store.cc:269] 0709 12:18:28.696856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.697056 (+   200us) service_pool.cc:224] Handling call
0709 12:18:30.370364 (+1673308us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54002abcd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:30.372166 (+  1802us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878623616}
I20260709 12:18:30.381474 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.393858 25786 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
W20260709 12:18:30.402328 26309 leader_election.cc:336] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.877s (SENT)
I20260709 12:18:30.196550 28206 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:30.171185 28028 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:30.184475 28188 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.409979 25633 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.147804 28269 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:30.411217 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:30.361351 28220 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.028s	user 0.017s	sys 0.007s
W20260709 12:18:30.354604 28284 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
W20260709 12:18:30.354029 28291 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 12:18:30.461268 28279 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.359s	user 0.000s	sys 0.001s
I20260709 12:18:30.140345 28253 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:30.461848 28028 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:30.445067 25630 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.446295 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.462734 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 699) took 1147 ms. Trace:
I20260709 12:18:30.196743 28287 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.210s
I20260709 12:18:30.463092 26343 rpcz_store.cc:276] 0709 12:18:29.315443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.316107 (+   664us) service_pool.cc:224] Handling call
0709 12:18:29.316528 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.437656 (+121128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.437691 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:29.451114 (+ 13423us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b2c0009f618. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:18:29.451638 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:18:29.642998 (+191360us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:29.643048 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.249587 (+606539us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54004970d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:30.249670 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.462468 (+212798us) raft_consensus.cc:1685] finished
0709 12:18:30.462504 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.462651 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280670848,"thread_start_us":190932,"threads_started":1,"wal-append.queue_time_us":512515}
W20260709 12:18:30.460664 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58402.2us
I20260709 12:18:30.456274 28186 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:30.465227 28139 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:30.465234 26309 leader_election.cc:336] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.888s (SENT)
I20260709 12:18:30.466347 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:30.468025 26309 leader_election.cc:336] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.945s (SENT)
I20260709 12:18:30.469185 28028 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:30.469337 26309 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.754s (SENT)
I20260709 12:18:30.469798 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:30.229049 28210 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.217s
I20260709 12:18:30.470530 28286 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.471011 28220 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:30.471092 28286 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:30.473170 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 280279us
I20260709 12:18:30.268821 28213 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:30.406355 28288 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.479478 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.093s (SENT)
I20260709 12:18:30.476649 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.486370 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:30.486624 25633 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 2.
I20260709 12:18:30.353258 27923 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 12:18:30.487995 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.441582 28285 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:30.494127 25630 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:30.496743 26309 leader_election.cc:336] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.152s (SENT)
I20260709 12:18:30.497309 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:30.493427 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 293921us
I20260709 12:18:30.482172 25636 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.444393 25634 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:30.286890 28106 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:30.487872 28283 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.631s	user 0.000s	sys 0.002s
I20260709 12:18:30.524140 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.524809 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.526978 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:30.445041 28024 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:18:30.458142 28295 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:30.487636 28220 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:30.490744 28091 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:30.522701 26340 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.466s	user 0.013s	sys 0.017s
I20260709 12:18:30.476331 28051 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:18:30.485702 28260 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:30.555958 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 384118us
I20260709 12:18:30.543713 25636 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:18:30.545926 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.559130 25634 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:30.490114 28288 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:30.528117 28210 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:18:30.537246 28218 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.441s	user 0.026s	sys 0.002s
I20260709 12:18:30.548580 25635 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:30.540210 28028 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:30.587807 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.588656 25634 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.553399 27936 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:30.541960 26309 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.228s (SENT)
I20260709 12:18:30.591799 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:30.592031 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50123.5us
I20260709 12:18:30.593093 25635 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:30.549355 28295 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:30.586520 25534 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 3, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:30.632082 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.556146 26340 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:30.580327 28210 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:30.584748 28219 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:30.630440 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.584164 28246 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:30.613875 28275 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.598s	user 0.001s	sys 0.000s
I20260709 12:18:30.659351 25632 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:30.583050 28206 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:18:30.661445 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.665714 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.666327 25630 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:30.585141 28218 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.614362 27923 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:30.666188 28260 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:30.680285 25631 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.656185 28295 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:30.666857 28028 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.666859 28246 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.664701 26340 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 694) took 1933 ms (client timeout 1579 ms). Trace:
I20260709 12:18:30.683060 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.671350 28188 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:30.683843 26340 rpcz_store.cc:269] 0709 12:18:28.726655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.763148 (+ 36493us) service_pool.cc:224] Handling call
0709 12:18:30.659790 (+1896642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.700409 28028 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:30.688313 28196 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.722572 28196 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:30.700640 28246 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.725270 28196 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:30.726465 28246 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:30.687292 26338 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:30.730406 28188 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:30.732331 28218 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:30.733042 28106 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:30.740628 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2123) took 1803 ms. Trace:
W20260709 12:18:30.742271 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.741266 25710 rpcz_store.cc:276] 0709 12:18:28.936756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.945688 (+  8932us) service_pool.cc:224] Handling call
0709 12:18:28.946121 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.124337 (+178216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.124385 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:29.157024 (+ 32639us) log.cc:844] Serialized 26 byte log entry
0709 12:18:29.394866 (+237842us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:29.394904 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.394970 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.740382 (+1345412us) raft_consensus.cc:1685] finished
0709 12:18:30.740404 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.740560 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66754,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":15741,"thread_start_us":333649,"threads_started":2,"wal-append.queue_time_us":432157}
I20260709 12:18:30.744987 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.677438 27923 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.593119 28285 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:30.679580 28274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.008s	sys 0.019s
W20260709 12:18:30.758131 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.759189 27936 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:30.762939 27936 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:30.763682 27534 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.495s	user 0.019s	sys 0.011s
I20260709 12:18:30.764431 25782 raft_consensus.cc:606] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:30.764902 25782 raft_consensus.cc:614] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:30.765494 27936 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:30.768905 28297 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:18:30.687022 28210 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.772167 28210 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:30.756403 28274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.009s	sys 0.019s
I20260709 12:18:30.775482 28274 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:30.776561 28274 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:30.740530 28188 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:30.781680 25630 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:30.787302 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:30.789580 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:30.807219 25786 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:30.790390 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:30.808936 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:30.809590 25787 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:30.792058 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:30.790782 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.802536 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:30.811825 25638 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.812343 25780 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:30.813294 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:30.813899 25780 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:30.785650 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.792852 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:30.839514 25785 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.003s	sys 0.011s
W20260709 12:18:30.840152 25785 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.003s	sys 0.011s
I20260709 12:18:30.815830 25784 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:30.848114 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.849115 25637 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.855121 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.856316 25636 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.859860 25780 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
W20260709 12:18:30.793977 28229 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.677s	user 0.023s	sys 0.004s
I20260709 12:18:30.861553 28229 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.862840 25707 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:30.863746 28229 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:30.550108 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 395417us
I20260709 12:18:30.874660 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:30.880744 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.881472 25630 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
W20260709 12:18:30.882511 25629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.002s	sys 0.046s
I20260709 12:18:30.884109 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 536) took 1277 ms. Trace:
W20260709 12:18:30.592612 26309 leader_election.cc:336] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.244s (SENT)
I20260709 12:18:30.893340 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:30.940452 25710 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
W20260709 12:18:30.782423 27952 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.970s	user 0.004s	sys 0.005s
I20260709 12:18:30.797053 28188 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:30.944757 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 72714.1us
W20260709 12:18:30.955508 25751 leader_election.cc:336] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.997s (SENT)
I20260709 12:18:30.946516 25708 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:18:30.942879 28258 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.070s	sys 0.092s
I20260709 12:18:30.956173 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:30.791275 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:30.951967 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:30.952171 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:30.962668 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:30.963356 25638 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:30.798564 27923 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:30.971204 25784 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:30.977595 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:30.978359 25709 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:30.881983 28191 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.957758 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 365148us
I20260709 12:18:30.958683 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:30.987468 25780 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
W20260709 12:18:30.959339 25602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.009s	sys 0.006s
I20260709 12:18:30.854573 28028 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:30.578238 28213 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:30.955253 28220 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:30.963851 28258 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.070s	sys 0.092s
W20260709 12:18:31.000929 26308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.020s	sys 0.067s
W20260709 12:18:31.008636 26308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.020s	sys 0.067s
W20260709 12:18:31.008951 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 761836us
W20260709 12:18:31.001353 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.047s	sys 0.040s
W20260709 12:18:31.001821 25779 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.032s	sys 0.043s
W20260709 12:18:31.010352 25779 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.033s	sys 0.043s
W20260709 12:18:31.010047 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.047s	sys 0.041s
I20260709 12:18:31.003268 25787 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
W20260709 12:18:30.993157 25602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.009s	sys 0.006s
I20260709 12:18:30.952634 28196 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:31.022310 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:30.990773 25778 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:30.978034 28268 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.005s	sys 0.009s
I20260709 12:18:31.027678 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:31.030305 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 85811.1us
W20260709 12:18:31.020054 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.021339 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:30.977965 27955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.000s	sys 0.021s
I20260709 12:18:31.007030 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:31.034026 25783 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:31.037050 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 81595.3us
W20260709 12:18:31.038748 25751 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.588s (SENT)
I20260709 12:18:31.039119 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.039004 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.039228 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.040720 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:31.025427 28028 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:18:31.035349 27987 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:18:31.047711 28028 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:31.040813 28001 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:18:31.048096 28028 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:31.041194 28234 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:18:31.048975 28028 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.041596 28220 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:31.049971 28220 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:31.050398 28089 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.002s
I20260709 12:18:31.050333 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:31.051379 25709 raft_consensus.cc:2410] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:31.050930 28089 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.053601 25706 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:18:31.055524 25751 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.565s (SENT)
W20260709 12:18:31.032624 28268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.005s	sys 0.010s
I20260709 12:18:31.058205 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.042716 25636 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
W20260709 12:18:31.059062 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 471787us
W20260709 12:18:31.035720 27955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.000s	sys 0.021s
I20260709 12:18:31.072793 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:31.064190 25787 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:31.072989 25781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.001s	sys 0.025s
W20260709 12:18:31.074939 25781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.001s	sys 0.025s
I20260709 12:18:30.994917 28191 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.061520 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.031351 28229 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.074357 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:31.030880 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:31.090006 25711 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:31.028815 28270 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:18:31.026975 28196 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.027143 28253 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 12:18:31.204852 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.207342 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.085062 25677 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.037s	sys 0.042s
I20260709 12:18:31.050411 28220 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.208909 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:31.221691 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 391) took 1068 ms. Trace:
W20260709 12:18:31.148276 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 75873.2us
W20260709 12:18:31.221094 25677 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.038s	sys 0.046s
I20260709 12:18:31.224231 25781 rpcz_store.cc:276] 0709 12:18:30.153375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.153554 (+   179us) service_pool.cc:224] Handling call
0709 12:18:30.153937 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.154200 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:30.551836 (+397636us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.221255 (+669419us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:31.221324 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:31.221381 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.221446 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.221608 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":605938,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":1189,"thread_start_us":1038695,"threads_started":2}
W20260709 12:18:31.095337 25751 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.829s (SENT)
I20260709 12:18:31.127434 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:31.237214 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 141928us
I20260709 12:18:31.118415 25787 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:18:31.148511 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 66082.9us
I20260709 12:18:31.042840 27923 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:31.063966 28210 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:31.201960 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 389) took 1064 ms. Trace:
W20260709 12:18:31.097955 25604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.001s	sys 0.011s
I20260709 12:18:31.075053 27952 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:31.150051 26344 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:31.260138 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 388) took 1122 ms. Trace:
I20260709 12:18:31.070775 27955 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:31.225312 25676 leader_election.cc:400] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:31.264600 25676 leader_election.cc:403] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:18:31.241003 25709 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:31.191402 25784 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:18:31.244885 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 117599us
W20260709 12:18:31.225411 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 684521us
I20260709 12:18:31.269961 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.270133 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.270908 25638 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:31.260722 25779 rpcz_store.cc:276] 0709 12:18:30.137478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.137617 (+   139us) service_pool.cc:224] Handling call
0709 12:18:30.179811 (+ 42194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.180109 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:31.010814 (+830705us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.259732 (+248918us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:31.259792 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:31.259839 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.259915 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.260069 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39252,"prepare.queue_time_us":581827,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":906850,"threads_started":2}
I20260709 12:18:31.282784 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:31.247843 25604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.013s	sys 0.018s
W20260709 12:18:31.251318 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 229060us
I20260709 12:18:31.205940 28219 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.247915 25785 rpcz_store.cc:276] 0709 12:18:30.137771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.137900 (+   129us) service_pool.cc:224] Handling call
0709 12:18:30.162539 (+ 24639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.162813 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:30.840591 (+677778us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.201389 (+360798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:31.201490 (+   101us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:31.201563 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.201665 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.201871 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19407,"prepare.queue_time_us":598862,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":22635,"spinlock_wait_cycles":21724928,"thread_start_us":976871,"threads_started":2}
I20260709 12:18:31.258442 25787 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:31.116778 28028 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:31.290866 26309 leader_election.cc:336] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.792s (SENT)
I20260709 12:18:31.296391 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:31.292001 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.294265 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 69118.7us
W20260709 12:18:31.297976 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.310303 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.311519 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:31.312179 26343 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:31.301478 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.318504 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.322355 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:31.324393 26309 leader_election.cc:336] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.829s (SENT)
I20260709 12:18:31.324863 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.325181 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.325367 25677 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.260377 27952 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.319172 25636 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
W20260709 12:18:31.330420 26309 leader_election.cc:336] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.101s (SENT)
I20260709 12:18:31.339370 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.265731 28285 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:31.351532 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:31.341480 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75245.8us
I20260709 12:18:31.246711 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:31.232625 28229 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:31.352015 25751 leader_election.cc:336] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.106s (SENT)
I20260709 12:18:31.365887 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.366437 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:31.354326 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:31.115357 28264 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.189s
I20260709 12:18:31.357208 25779 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:31.352089 25637 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:18:31.365430 26309 leader_election.cc:336] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.209s (SENT)
I20260709 12:18:31.380239 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.367940 25636 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:18:31.373560 26344 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:31.374984 25785 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:31.376116 25751 leader_election.cc:336] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.911s (SENT)
I20260709 12:18:31.381083 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:31.382074 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:31.289830 28091 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:31.393478 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.333153 28268 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:31.351982 27952 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.265341 28299 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 12:18:31.365283 28304 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:31.334069 28260 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:31.356478 28258 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:31.351615 28285 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:31.370541 28306 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:31.399998 28317 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:18:31.397295 28307 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:31.354920 28219 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.408771 28298 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:31.412570 28234 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.475327 28234 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.477265 28234 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:31.457960 28303 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
W20260709 12:18:31.477891 25708 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.000s
I20260709 12:18:31.415722 27936 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:31.478907 28314 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
W20260709 12:18:31.479858 28309 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.233s	user 0.000s	sys 0.000s
I20260709 12:18:31.435703 28285 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:31.473776 28329 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:31.474546 28219 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.478236 25708 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:31.484961 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.505089 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.463248 28268 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:31.462673 28260 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:31.465291 28307 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:31.488705 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 166426us
I20260709 12:18:31.462472 28206 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:31.520411 28206 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:31.520944 28206 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:31.489023 28298 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:31.484206 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 158818us
I20260709 12:18:31.523206 28298 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.523787 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.526243 28260 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:31.526648 28260 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.527029 28260 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:31.467881 27955 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.521847 28206 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.532667 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:31.536659 28340 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:31.537254 28340 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:31.538090 28340 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:31.459807 28089 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:31.538444 28191 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:31.545666 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2060) took 2221 ms. Trace:
I20260709 12:18:31.547645 25707 rpcz_store.cc:276] 0709 12:18:29.324582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.324749 (+   167us) service_pool.cc:224] Handling call
0709 12:18:31.545440 (+2220691us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54003ac854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:31.545577 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3382282112}
I20260709 12:18:31.548730 28288 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:31.549155 25707 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:31.409008 28091 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.551538 28229 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.552510 28091 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:31.515506 28268 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.553488 28268 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.482316 28258 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:31.557878 28258 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.436362 28304 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.560348 28258 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.499205 28318 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:31.561347 28318 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.561488 28304 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:31.561798 28318 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.567890 28267 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:31.568365 28267 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:31.569242 28267 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.570261 28267 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:31.570802 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1900) took 1072 ms. Trace:
I20260709 12:18:31.571322 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:31.571973 25783 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.000s
I20260709 12:18:31.572342 25783 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:31.571216 25631 rpcz_store.cc:276] 0709 12:18:30.498835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.523876 (+ 25041us) service_pool.cc:224] Handling call
0709 12:18:31.570616 (+1046740us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540021fa54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:31.570743 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357137664}
I20260709 12:18:31.576177 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.580097 28290 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:31.580493 28323 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 12:18:31.580119 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.886639 25629 rpcz_store.cc:276] 0709 12:18:29.629598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.629758 (+   160us) service_pool.cc:224] Handling call
0709 12:18:29.651344 (+ 21586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.651552 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:30.369399 (+717847us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.883817 (+514418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.883856 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:30.883894 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.883959 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.884049 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":931938,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":129,"thread_start_us":1207246,"threads_started":2}
W20260709 12:18:31.536934 28313 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.878s	user 0.002s	sys 0.008s
I20260709 12:18:31.587353 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:31.588241 25707 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:31.592769 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.594740 28287 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:31.607059 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:31.595230 28287 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:31.611059 28295 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.903s	user 0.008s	sys 0.004s
W20260709 12:18:31.652668 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55945.2us
I20260709 12:18:31.458638 28306 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.695398 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.699309 28091 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.699613 28089 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:31.699926 28091 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.702317 28091 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.706811 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.615906 28229 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:31.707232 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.707479 25631 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
W20260709 12:18:31.652709 28317 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:31.618876 28287 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:31.720959 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.738374 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:31.738166 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:31.739148 25634 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:31.700372 28089 raft_consensus.cc:2764] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:18:31.741659 28089 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:31.744292 28089 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.745638 28089 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:31.694010 28340 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.739012 25629 raft_consensus.cc:2376] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5. Re-sending same reply.
I20260709 12:18:31.756987 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1901) took 1221 ms. Trace:
I20260709 12:18:31.757823 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.761180 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:31.759518 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 378512us
I20260709 12:18:31.762503 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.765202 25631 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:31.757543 25633 rpcz_store.cc:276] 0709 12:18:30.535774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.536053 (+   279us) service_pool.cc:224] Handling call
0709 12:18:30.536382 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.536705 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.536734 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:30.540159 (+  3425us) log.cc:844] Serialized 26 byte log entry
0709 12:18:31.028241 (+488082us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:31.070297 (+ 42056us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.089356 (+ 19059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.089426 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.756693 (+667267us) raft_consensus.cc:1685] finished
0709 12:18:31.756726 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.756911 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18758,"prepare.queue_time_us":42323,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":81952,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":45083,"spinlock_wait_cycles":1866496,"thread_start_us":442815,"threads_started":1,"wal-append.queue_time_us":647674}
I20260709 12:18:31.769296 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.735260 28229 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:31.781764 25631 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
W20260709 12:18:31.787699 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.764894 28188 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.796s	user 0.009s	sys 0.004s
I20260709 12:18:31.750460 28340 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:31.798069 25634 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
W20260709 12:18:31.812203 25738 heartbeater.cc:646] Failed to heartbeat to 127.23.217.126:36567 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.23.217.126:36567 timed out after 15.000s (SENT)
I20260709 12:18:31.814291 25630 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
W20260709 12:18:31.815336 28339 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:18:31.798003 28051 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:31.817111 28335 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:18:31.825238 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2009) took 1021 ms. Trace:
I20260709 12:18:31.764335 28139 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:31.825639 25786 rpcz_store.cc:276] 0709 12:18:30.803524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.849731 (+ 46207us) service_pool.cc:224] Handling call
0709 12:18:30.850132 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:30.850599 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.850638 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:30.851505 (+   867us) log.cc:844] Serialized 52 byte log entry
0709 12:18:31.240999 (+389494us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:31.246911 (+  5912us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.247177 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.247259 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.825045 (+577786us) raft_consensus.cc:1685] finished
0709 12:18:31.825065 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.825186 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19310,"prepare.run_wall_time_us":6880,"raft.queue_time_us":12761,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":14852,"spinlock_wait_cycles":40123008,"thread_start_us":272448,"threads_started":1,"wal-append.queue_time_us":877100}
I20260709 12:18:31.827548 27952 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.707130 28295 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:31.830605 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.764144 28024 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:31.830960 28188 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.831255 28260 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.797227 28304 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:31.835422 28220 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.595s	user 0.010s	sys 0.000s
W20260709 12:18:31.835487 25778 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.799s	user 0.005s	sys 0.005s
I20260709 12:18:31.836313 28268 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:31.813278 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50851us
I20260709 12:18:31.839072 28229 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.840003 28270 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:31.836273 28089 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.842886 28306 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:31.844220 26340 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.695s	user 0.005s	sys 0.005s
W20260709 12:18:31.846608 28028 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
I20260709 12:18:31.852591 28024 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.859136 28295 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:31.859638 25713 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:31.852011 28285 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:31.803323 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.863916 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50156.3us
I20260709 12:18:31.875615 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.872710 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:31.901770 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:31.880414 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.804656 28298 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:31.892791 25705 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:31.904435 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:31.906219 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 166315us
I20260709 12:18:31.904909 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:31.904592 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:31.908154 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:31.852262 25710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.004s	sys 0.026s
W20260709 12:18:31.908969 25710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.004s	sys 0.026s
I20260709 12:18:31.910140 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.910347 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:31.910873 25636 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:31.910799 25783 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:31.871266 27952 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:31.910059 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:31.911651 25710 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.908356 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.912248 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.913295 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:31.914094 25634 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:31.915158 25784 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:18:31.915313 25629 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:31.915514 25784 pending_rounds.cc:85] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 5
I20260709 12:18:31.915645 25629 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:31.915755 25784 pending_rounds.cc:107] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:18:31.916024 25784 raft_consensus.cc:2889] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:31.916397 25629 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.917313 25629 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:31.899621 28318 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.920164 28318 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.905428 28301 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:31.905712 28298 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.921384 28298 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:31.922593 28345 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:31.873725 28220 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:31.924479 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2063) took 1047 ms. Trace:
I20260709 12:18:31.924770 25631 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:31.924898 25705 rpcz_store.cc:276] 0709 12:18:30.877097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.877264 (+   167us) service_pool.cc:224] Handling call
0709 12:18:30.952951 (+ 75687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:31.905820 (+952869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.924208 (+ 18388us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54003c82d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:31.924404 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160904064}
I20260709 12:18:31.905944 28306 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:31.925834 28220 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.877292 28208 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:18:31.877162 25778 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:31.927003 25781 raft_consensus.cc:2410] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:31.927201 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:31.895476 26340 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:31.889760 28260 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.926098 28273 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:31.927925 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 52410.2us
I20260709 12:18:31.948325 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:31.925981 28306 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.931565 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.932597 25784 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:31.953737 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.932910 25676 connection.cc:256] RPC call timeout handler was delayed by -2.6440313570001308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
I20260709 12:18:31.936246 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:31.936679 25631 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:31.937602 25633 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:31.955410 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:31.938166 26339 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:31.949429 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.949924 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:31.956194 25634 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:31.954326 25676 connection.cc:256] RPC call timeout handler was delayed by -2.5387313570001879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 12:18:31.961169 25676 connection.cc:256] RPC call timeout handler was delayed by -2.5387313570001879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 12:18:31.961516 25676 connection.cc:256] RPC call timeout handler was delayed by -2.5234313570001632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
W20260709 12:18:31.961817 25676 connection.cc:256] RPC call timeout handler was delayed by -2.4802313570000933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
W20260709 12:18:31.962110 25676 connection.cc:256] RPC call timeout handler was delayed by -2.4262313570002334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 12:18:31.962394 25676 connection.cc:256] RPC call timeout handler was delayed by -2.4019313570001941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:18:31.962699 25676 connection.cc:256] RPC call timeout handler was delayed by -2.3434313570000995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
W20260709 12:18:31.962982 25676 connection.cc:256] RPC call timeout handler was delayed by -2.2642313570001988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:18:31.963261 25676 connection.cc:256] RPC call timeout handler was delayed by -2.2462313570001697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
W20260709 12:18:31.963551 25676 connection.cc:256] RPC call timeout handler was delayed by -2.1481313570002385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
W20260709 12:18:31.963832 25676 connection.cc:256] RPC call timeout handler was delayed by -2.0275313570000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
W20260709 12:18:31.964108 25676 connection.cc:256] RPC call timeout handler was delayed by -1.4740313570000581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2859s for a response
W20260709 12:18:31.964395 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3237313570000424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3026s for a response
W20260709 12:18:31.964674 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2958313570002247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3057s for a response
W20260709 12:18:31.964949 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2823313570002028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30720000000000003s for a response
W20260709 12:18:31.965230 25676 connection.cc:256] RPC call timeout handler was delayed by -1.13383135700019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32370000000000004s for a response
I20260709 12:18:31.965821 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:31.903218 28346 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.971297 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:31.971472 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:31.973454 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.974035 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.974553 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.974748 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.929697 28337 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:31.973366 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.983595 25785 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:31.984706 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:31.985770 25785 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:18:31.977733 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:31.986703 25780 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:31.978410 26342 raft_consensus.cc:2368] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:31.933687 28225 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:31.980080 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:31.975883 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:31.987645 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.934382 28028 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:31.946421 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2003) took 1155 ms. Trace:
I20260709 12:18:32.022037 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:31.969341 28220 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:31.946379 28273 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:31.969074 28306 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:31.952284 28191 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.967368 28270 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:18:31.990252 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:32.024418 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.978924 27952 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:18:32.154016 25710 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:32.364308 25636 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:18:32.444401 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 468549us
I20260709 12:18:32.443498 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:32.222662 25629 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:32.451189 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:32.136124 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:31.993170 28295 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:32.455374 28295 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:32.456784 28295 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.194725 28258 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:32.462427 28258 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:32.043635 28260 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:32.465627 28260 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:32.015350 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:32.466986 28260 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:32.246874 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 222839us
I20260709 12:18:31.952497 28139 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:32.467986 26337 raft_consensus.cc:2368] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:32.523244 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:32.484592 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2140) took 1404 ms. Trace:
W20260709 12:18:32.026158 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50709.3us
I20260709 12:18:32.472126 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:32.141103 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.447204 25634 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
W20260709 12:18:32.570307 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.569343 25785 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:32.447738 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:32.572244 26341 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:32.568362 25709 rpcz_store.cc:276] 0709 12:18:31.080420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.284478 (+204058us) service_pool.cc:224] Handling call
0709 12:18:31.333674 (+ 49196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:31.796790 (+463116us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:31.864496 (+ 67706us) log.cc:844] Serialized 14 byte log entry
0709 12:18:32.259133 (+394637us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:32.298285 (+ 39152us) log.cc:844] Serialized 26 byte log entry
0709 12:18:32.298509 (+   224us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:32.298554 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.298661 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:32.484277 (+185616us) raft_consensus.cc:1685] finished
0709 12:18:32.484308 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.484502 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15058,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":216832,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":202,"thread_start_us":824300,"threads_started":2,"wal-append.queue_time_us":597165}
I20260709 12:18:31.990202 28342 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:32.386955 28304 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.526s	user 0.000s	sys 0.012s
I20260709 12:18:32.590505 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:32.561671 28312 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:32.219928 28332 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 12:18:32.134161 28324 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.625s	user 0.000s	sys 0.003s
W20260709 12:18:32.031594 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 65873.3us
I20260709 12:18:32.592306 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:32.597090 25787 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:32.049667 28345 raft_consensus.cc:2764] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:32.470719 28341 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:32.447984 28220 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.044274 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:32.603313 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:32.604095 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:31.993247 28350 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.525818 28208 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:32.235636 28188 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:32.127744 28285 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:32.129225 28089 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:31.989235 28347 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.621s
I20260709 12:18:32.608229 25629 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:32.608255 25634 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.609462 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:32.198894 28298 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:31.979576 28346 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.021908 25778 rpcz_store.cc:276] 0709 12:18:30.790749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.790922 (+   173us) service_pool.cc:224] Handling call
0709 12:18:31.932409 (+1141487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.619145 28208 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:32.209040 28287 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.625376 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:32.625834 25780 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:32.600361 28304 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:32.629815 25783 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:32.529906 28191 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.610096 28295 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:32.614701 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:32.617295 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:32.608193 28220 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:32.632409 25785 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:32.632686 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:32.612629 28139 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.635658 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:32.636502 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:32.626236 26338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 473) took 1635 ms (client timeout 2172 ms). Trace:
I20260709 12:18:32.612821 28268 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:32.638221 28188 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:32.641798 25786 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:32.622645 28310 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:32.644052 25784 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:32.601756 28312 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:32.638525 28358 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:32.638831 28089 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:32.648458 25634 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:18:32.597826 28342 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.615188 28350 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:32.642131 26340 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:32.614094 28267 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:18:32.640206 28139 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.654250 28089 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:32.666838 28089 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:32.658167 28188 raft_consensus.cc:2764] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:32.667873 28298 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:32.673349 28338 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:32.673825 28338 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:32.656492 25677 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:32.663163 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 21) took 3340 ms. Trace:
I20260709 12:18:32.663833 28346 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:32.663841 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1881) took 2813 ms. Trace:
I20260709 12:18:32.653493 25629 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:32.678503 25782 rpcz_store.cc:276] 0709 12:18:29.322991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.323136 (+   145us) service_pool.cc:224] Handling call
0709 12:18:32.662971 (+3339835us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54002b5054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5989853 00007f29ba95760a 00007f29b3edd7d4 00007f29b3edd762 00007f29b3edd691 00007f29b3edd60c 00007f29b3edc8c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:32.663078 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3159189376}
I20260709 12:18:32.679898 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:32.702857 28188 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:32.704169 28365 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:32.657657 28208 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.719916 28188 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:32.721035 28188 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.722843 28298 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:32.723415 28298 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:32.723845 28320 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.235s	user 0.001s	sys 0.019s
W20260709 12:18:32.732831 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 59250.3us
I20260709 12:18:32.734750 25711 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:32.678730 25632 rpcz_store.cc:276] 0709 12:18:29.850094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.208490 (+358396us) service_pool.cc:224] Handling call
0709 12:18:30.270678 (+ 62188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.665946 (+395268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.663640 (+1997694us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b540020b554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:32.663786 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893114112}
I20260709 12:18:32.701864 28338 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:32.640354 26338 rpcz_store.cc:269] 0709 12:18:30.990443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.993920 (+  3477us) service_pool.cc:224] Handling call
0709 12:18:32.625237 (+1631317us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aeee 000000000036eb39 00007f29ba8de1e0 00007f29ba9a70c4 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:32.626168 (+   931us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6363008}
I20260709 12:18:32.757375 25712 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:32.764427 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 292407us
W20260709 12:18:32.765007 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 128555us
I20260709 12:18:32.751686 28191 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:32.768260 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:32.767378 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:32.770213 25712 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:32.772311 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:32.748400 28310 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:18:32.762779 28338 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:32.772742 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:32.759675 28295 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:32.767025 28337 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.005s
I20260709 12:18:32.796103 25709 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:32.802205 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:32.808781 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.806313 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:32.747346 28298 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.800868 25637 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:32.795814 28343 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:32.794984 28139 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:32.801630 28337 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:32.825104 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 56942.6us
W20260709 12:18:32.830119 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57452.8us
I20260709 12:18:32.830389 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:32.826741 28286 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:32.831866 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:32.813198 28234 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:32.837320 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:32.812937 28312 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:32.818248 28346 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:32.828439 27952 raft_consensus.cc:993] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:18:32.826802 28350 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:32.824712 28342 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:32.853183 28353 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:32.846704 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:32.822238 28260 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:32.859690 28214 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:32.883867 28214 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:32.860769 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:32.886381 28350 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:32.886775 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:32.887158 28350 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:32.887854 28310 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:32.888805 28350 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:32.855078 28374 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:32.897795 28220 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:32.900247 28295 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.902052 25712 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:32.859158 28341 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:32.909806 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:32.890744 27952 raft_consensus.cc:1081] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:18:32.862306 28346 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:32.911516 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:32.864281 27955 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 12:18:32.919515 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72894.3us
I20260709 12:18:32.918713 25709 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:32.922569 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:32.924924 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:32.927942 26339 raft_consensus.cc:2368] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:32.935825 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:32.921708 28342 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:32.938850 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:32.943377 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:32.946108 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:32.946965 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:32.947168 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:32.934577 28208 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:32.928944 28341 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:32.948002 25634 raft_consensus.cc:2376] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 3. Re-sending same reply.
I20260709 12:18:32.922021 28353 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:18:32.954761 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 539) took 1649 ms. Trace:
I20260709 12:18:32.955125 25638 rpcz_store.cc:276] 0709 12:18:31.305587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.306944 (+  1357us) service_pool.cc:224] Handling call
0709 12:18:32.954099 (+1647155us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5800000c98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:32.954285 (+   186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.954447 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:32.954475 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:32.954535 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:32.954566 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.954621 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.954710 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25808256}
I20260709 12:18:32.955940 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:32.956661 25629 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:32.958070 28024 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:32.959257 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2019) took 1689 ms. Trace:
I20260709 12:18:32.959654 25781 rpcz_store.cc:276] 0709 12:18:31.269516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.269664 (+   148us) service_pool.cc:224] Handling call
0709 12:18:32.959036 (+1689372us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400099f54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:32.959184 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508862336}
I20260709 12:18:32.948096 28369 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:18:32.942847 28310 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:32.949486 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:32.944970 28191 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:32.953383 28208 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:32.948167 28342 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:32.965162 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:32.968209 25779 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:32.937211 27955 consensus_queue.cc:1243] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:18:32.970012 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47571.3us
I20260709 12:18:32.963806 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:32.983807 25638 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:32.973415 28290 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:18:32.977449 28220 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:32.977979 28346 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:32.930631 28312 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:18:32.979080 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:32.988830 25629 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:32.991011 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:32.991192 28342 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:32.971107 28310 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.010993 28310 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:33.011827 28306 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.014372 28306 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:33.015599 28208 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.019516 28001 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:18:32.984158 25635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 533) took 3638 ms (client timeout 1903 ms). Trace:
W20260709 12:18:33.022204 25635 rpcz_store.cc:269] 0709 12:18:29.345606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.345759 (+   153us) service_pool.cc:224] Handling call
0709 12:18:32.983982 (+3638223us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001c5cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:32.984091 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2673728256}
I20260709 12:18:32.985769 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:33.008641 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.984112 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:33.030542 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:33.025790 25632 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:33.034381 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:33.035331 25638 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:33.034333 26307 connection.cc:256] RPC call timeout handler was delayed by -2.0141672800000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
I20260709 12:18:33.033383 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:33.035898 26307 connection.cc:256] RPC call timeout handler was delayed by -1.5677176850001615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
I20260709 12:18:33.037930 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:33.043229 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.044574 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:33.045266 25635 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:33.022073 28001 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:33.060811 25781 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:33.046708 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:32.973999 28089 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.064373 25781 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:33.065428 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.066272 25787 raft_consensus.cc:2410] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:33.042402 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:33.036630 28260 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:33.032114 28188 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:33.070853 25676 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:33.076522 25638 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:33.080565 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.081897 25711 raft_consensus.cc:2410] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:18:33.036724 28374 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.056190 28287 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:33.036882 28024 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.059327 28186 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:33.090096 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:33.091673 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.048956 28258 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.094869 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.088127 25638 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:33.092062 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:33.098451 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.118858 25705 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:33.119715 28186 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:33.120534 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.121130 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2027) took 1190 ms. Trace:
I20260709 12:18:33.119285 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.121583 25783 rpcz_store.cc:276] 0709 12:18:31.931085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.931261 (+   176us) service_pool.cc:224] Handling call
0709 12:18:33.120982 (+1189721us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b5400098b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:33.121077 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1556550016}
I20260709 12:18:33.105858 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.122123 25635 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:33.103771 25710 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:33.085145 28347 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:33.082394 28376 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.112128 28089 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.069062 28346 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.068933 28338 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.069135 28260 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:33.068893 28290 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.088682 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.151790 26340 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:33.153141 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.081614 28286 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.120725 28186 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.088124 28001 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.159349 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.162513 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:33.163262 26340 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:33.160254 25712 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
W20260709 12:18:33.109408 28359 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.910s	user 0.014s	sys 0.005s
I20260709 12:18:33.123159 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.081367 28341 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:33.134896 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.150856 25782 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:18:33.168169 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:33.169941 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:33.150381 28290 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.113101 28024 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.153532 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2142) took 2037 ms. Trace:
I20260709 12:18:33.155946 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.174829 25629 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:33.122131 28258 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:33.147970 28260 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:33.165918 25638 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:33.185114 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.185768 25706 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:33.174436 25708 rpcz_store.cc:276] 0709 12:18:31.116378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.510005 (+393627us) service_pool.cc:224] Handling call
0709 12:18:31.510733 (+   728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:31.577074 (+ 66341us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:31.609996 (+ 32922us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.890261 (+280265us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:32.675543 (+785282us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b2c000127c8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:18:32.675997 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 12:18:32.676141 (+   144us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:32.676177 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.676259 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:33.151430 (+475171us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b1800414e80. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:33.153245 (+  1815us) raft_consensus.cc:1685] finished
0709 12:18:33.153274 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.153447 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61983,"prepare.queue_time_us":32187,"prepare.run_cpu_time_us":20060,"prepare.run_wall_time_us":68530,"raft.queue_time_us":11573,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":4889,"spinlock_wait_cycles":17970688,"thread_start_us":191596,"threads_started":1,"wal-append.queue_time_us":296971}
I20260709 12:18:33.173346 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 482) took 1966 ms. Trace:
I20260709 12:18:33.177040 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1533) took 1229 ms. Trace:
I20260709 12:18:33.177322 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.177991 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2078) took 1608 ms. Trace:
I20260709 12:18:33.184511 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.153916 28338 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.190022 25712 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:33.189711 25713 rpcz_store.cc:276] 0709 12:18:31.569780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.846652 (+276872us) service_pool.cc:224] Handling call
0709 12:18:31.846954 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:31.900610 (+ 53656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.177704 (+1277094us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5400224cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:33.177918 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28728704}
I20260709 12:18:33.189411 25779 rpcz_store.cc:276] 0709 12:18:31.947544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.947746 (+   202us) service_pool.cc:224] Handling call
0709 12:18:32.960684 (+1012938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.982129 (+ 21445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.176853 (+194724us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400099f50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:33.176980 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328791424}
I20260709 12:18:33.166884 28196 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.172261 27936 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.148466 28347 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.195597 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.175886 28287 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:33.153784 28346 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:33.185019 28191 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:33.208287 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:33.209133 25713 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:33.210636 25708 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:33.212004 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.213641 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.163926 28286 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.204432 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:33.216225 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:33.217792 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.189649 28001 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:33.223923 25712 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:33.201186 27952 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:33.173934 28310 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.238107 28376 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.241803 28376 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.243808 28346 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:33.244010 28376 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.238255 28089 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.245024 28089 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.247257 28089 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.238317 28342 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:33.248159 28342 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.249642 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:33.250293 28342 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:33.250708 28220 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.236784 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:33.251314 28220 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.253679 28220 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.254252 28220 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:33.254694 28342 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.254769 28220 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:33.255719 28220 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:33.256462 25676 leader_election.cc:400] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:33.256814 25676 leader_election.cc:403] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:18:33.255208 28338 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:33.257496 28024 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.259954 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 545) took 1495 ms. Trace:
I20260709 12:18:33.260319 25630 rpcz_store.cc:276] 0709 12:18:31.764330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.766262 (+  1932us) service_pool.cc:224] Handling call
0709 12:18:31.814082 (+ 47820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:31.814938 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.259775 (+1444837us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b540021f050. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:33.259899 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37585664}
I20260709 12:18:33.263496 28286 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.188925 28258 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.266446 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:33.269163 28258 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.262581 28260 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.272337 28260 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.274789 26341 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:33.275972 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:33.276471 26341 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:33.276340 28273 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.277842 28273 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.199483 26335 rpcz_store.cc:276] 0709 12:18:31.207234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.222209 (+ 14975us) service_pool.cc:224] Handling call
0709 12:18:32.907980 (+1685771us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:32.908237 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.908506 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:32.972394 (+ 63888us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.157745 (+185351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:33.164734 (+  6989us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:33.173077 (+  8343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.173162 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.173289 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":698,"prepare.queue_time_us":63912,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":1407,"raft.queue_time_us":5923,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3860,"spinlock_wait_cycles":18695936,"thread_start_us":227898,"threads_started":2,"wal-append.queue_time_us":204202}
I20260709 12:18:33.280985 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:33.281591 26335 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.282655 28342 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:33.282934 28260 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.284690 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:33.285887 25676 leader_election.cc:400] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:33.286158 25676 leader_election.cc:403] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:33.279606 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.226274 28264 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:33.224642 28139 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.293566 25632 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:33.295806 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.296684 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:33.295017 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 234135us
I20260709 12:18:33.232834 28337 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.226995 28196 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.306382 28206 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.306962 28206 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.307109 28225 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:33.234119 28267 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:33.232852 28287 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:33.313926 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.314641 26343 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:33.311668 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.315542 25781 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:33.316917 25713 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:33.319020 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.319918 25779 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:33.323392 26341 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:33.297534 28342 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4.
I20260709 12:18:33.296875 28260 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:33.329694 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 112009us
W20260709 12:18:33.328624 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 191046us
I20260709 12:18:33.331514 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.332417 25785 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:18:33.349516 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 63733.7us
I20260709 12:18:33.350447 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:33.350677 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:33.351363 26341 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:33.355279 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:33.366693 25713 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:33.378517 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.379427 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:33.380764 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:33.381573 25638 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:33.381596 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.382263 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:33.382898 25634 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:33.382936 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.383613 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.384243 25638 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:33.379251 25782 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:33.385197 26341 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:33.386907 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.387734 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.391125 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.392447 25785 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.391125 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.393523 25636 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:33.398747 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:33.399971 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.400331 25636 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:33.400754 26343 raft_consensus.cc:2410] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:33.392277 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.402153 25780 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:33.405028 25634 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:33.407902 25781 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:33.410081 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.410413 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:33.412402 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.410820 25629 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:33.413650 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.352522 28260 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:33.419963 25634 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:18:33.385821 28337 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:33.427601 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:33.391600 28091 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.425035 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.424140 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:33.381480 28340 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.389174 28139 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:18:33.389221 28208 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:33.425011 28260 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:33.435701 25634 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:18:33.389806 26340 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:33.408996 28001 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:18:33.441015 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:33.428942 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 178609us
I20260709 12:18:33.479737 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.423345 28286 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.423193 28350 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:33.422397 25780 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:18:33.426036 28024 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.438055 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.459766 25785 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:33.429524 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.455173 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:33.579468 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.580693 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.431361 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.483098 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.485532 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.585661 25629 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:33.487612 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.490200 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:33.434710 28337 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:33.428755 28369 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:18:33.486471 28350 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.490828 28397 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:33.557287 28310 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:33.558342 28391 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:18:33.604121 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:33.561336 28394 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:33.619545 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:33.620357 28368 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:33.620819 28368 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:33.538364 28342 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 12:18:33.623462 28220 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.624630 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.585950 28214 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:33.625943 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.452091 28208 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.627034 28208 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.628912 28337 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:33.539904 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.631000 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.422981 27955 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 12:18:33.587141 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 97004us
I20260709 12:18:33.587177 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.491365 28089 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 12:18:33.593457 27952 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:33.541209 28091 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.494849 28298 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:18:33.548854 28139 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:33.433955 28287 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.565811 28312 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:33.583375 28267 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:33.539886 28290 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:33.603199 28338 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:33.563637 28024 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.616828 26307 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:33.617398 28258 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:33.586319 28196 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:33.632292 25780 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:33.649577 27952 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.632332 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:33.678994 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 46763.1us
W20260709 12:18:33.585268 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 111020us
I20260709 12:18:33.623409 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.681066 25787 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.682966 28024 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.639916 25783 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:33.687469 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.644694 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:33.689281 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72510.9us
I20260709 12:18:33.647099 25711 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
W20260709 12:18:33.689867 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 210240us
I20260709 12:18:33.633311 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:33.690340 26307 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:33.690672 25708 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:33.690992 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.590665 28347 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.693778 25783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:33.539361 28286 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.601186 28397 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:33.625810 28220 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.675002 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:33.687330 27952 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.587889 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.649444 28306 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:33.649925 28234 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.690083 25711 pending_rounds.cc:85] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 3
I20260709 12:18:33.729931 25711 pending_rounds.cc:107] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:18:33.669102 28139 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.691550 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:33.692638 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.621773 28368 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.695474 25783 raft_consensus.cc:606] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:33.697630 25787 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:33.700240 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.700865 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.701141 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:33.712128 25784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.022s	sys 0.019s
I20260709 12:18:33.719871 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 639) took 1719 ms. Trace:
W20260709 12:18:33.739604 25784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.023s	sys 0.019s
I20260709 12:18:33.739794 26342 rpcz_store.cc:276] 0709 12:18:32.000632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.009167 (+  8535us) service_pool.cc:224] Handling call
0709 12:18:33.674853 (+1665686us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:33.719802 (+ 44949us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14793088}
I20260709 12:18:33.676893 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.677713 28338 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.663367 28091 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.730321 25711 raft_consensus.cc:2889] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:33.602066 28350 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.693504 26340 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 5 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:33.739112 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.725773 26336 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:18:33.768275 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75711us
I20260709 12:18:33.710508 28196 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:18:33.781759 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:33.711653 28340 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:33.731206 28341 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.005s
I20260709 12:18:33.775966 25779 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
W20260709 12:18:33.765977 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 74479.5us
I20260709 12:18:33.777462 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.720966 28258 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:33.748203 28290 raft_consensus.cc:2764] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:33.779503 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:33.778612 25708 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:33.784175 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:33.788200 25779 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:33.787043 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:33.802431 28213 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:33.804623 28401 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:33.736277 28234 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.806712 26343 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:33.807942 25635 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:33.808408 28234 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:33.809628 26340 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:33.797470 28397 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:33.812776 28385 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.770s	user 0.002s	sys 0.007s
I20260709 12:18:33.785342 28368 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:33.814744 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.817821 28220 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:33.818094 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:33.797235 26335 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.003s
I20260709 12:18:33.819069 26335 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:33.820021 28024 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:33.820120 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:33.820649 28024 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:33.821111 28024 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:33.785893 28340 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.822892 28340 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:33.822664 28347 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:33.823505 28347 consensus_queue.cc:1243] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:33.824036 28347 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:33.824465 28347 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:33.825309 28347 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.779191 28298 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:33.826689 28298 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.823521 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.818296 28220 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:33.832152 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:33.835388 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51306.5us
I20260709 12:18:33.835702 26341 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:33.836084 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:33.830295 25782 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:33.732508 28306 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:33.773782 28273 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:33.776386 28405 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.793016 28341 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.783833 28267 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:33.794457 28258 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.843417 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.849151 28338 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:33.850033 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.872221 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:33.834461 28403 raft_consensus.cc:2764] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:33.839970 28220 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:33.876228 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:33.837373 28186 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:33.874709 25638 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:33.894124 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:33.875293 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 185025us
I20260709 12:18:33.844683 28273 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.841929 28089 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:33.895138 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 275651us
I20260709 12:18:33.896987 26343 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:33.843005 28410 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:33.905210 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:33.909969 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 112059us
I20260709 12:18:33.883373 28213 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:33.884382 28298 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.876753 28258 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:33.886587 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:33.887166 28341 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.916483 25638 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:33.893991 28186 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.873785 28411 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.896653 28310 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.927079 28310 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:18:33.929401 28310 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.903893 28273 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.932346 28139 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:33.937168 25752 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.873s (SENT) [suppressed 23 similar messages]
I20260709 12:18:33.939788 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:33.942054 28407 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:18:33.945856 26340 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:33.917177 28213 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.951958 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:33.956727 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.953051 28415 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.925765 28186 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:33.960098 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.962955 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.970300 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.970682 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:33.963505 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:33.923837 28298 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:33.964756 25781 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:33.970685 25779 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 5 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:18:33.973182 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:33.966048 28267 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:33.961396 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:33.960584 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:33.997210 28361 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.952s	user 0.008s	sys 0.000s
I20260709 12:18:34.015769 25633 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:34.017614 26344 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:34.029318 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 642) took 2002 ms. Trace:
I20260709 12:18:34.032507 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:34.040572 26339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 635) took 2073 ms (client timeout 2411 ms). Trace:
W20260709 12:18:34.041103 26339 rpcz_store.cc:269] 0709 12:18:31.967552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.968025 (+   473us) service_pool.cc:224] Handling call
0709 12:18:34.040315 (+2072290us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b540065c2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:34.040495 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827857792}
I20260709 12:18:34.043032 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:34.050689 26338 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:18:34.029922 26336 rpcz_store.cc:276] 0709 12:18:32.026824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.591024 (+564200us) service_pool.cc:224] Handling call
0709 12:18:32.591358 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.778207 (+1186849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.028966 (+250759us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000a37d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:34.029222 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2608162176}
I20260709 12:18:34.059412 28206 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:34.014158 25786 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:34.081935 25534 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 5, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:34.087435 25534 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 2 to 3, term changed from 2 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to <none>, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:34.089604 25534 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:33.970975 25638 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:33.925766 28410 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:33.961895 26340 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:34.114862 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 220833us
I20260709 12:18:34.115722 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:34.105450 25534 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:33.912899 28089 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.115725 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:34.121006 25534 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 5, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:33.971578 28286 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:33.965312 28213 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:34.115106 28214 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 12:18:34.121982 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 200517us
W20260709 12:18:34.125204 25532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 297) took 17464 ms (client timeout 15000 ms). Trace:
I20260709 12:18:34.128989 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:34.125149 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:34.125355 25534 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:18:34.116534 25778 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.000s
I20260709 12:18:34.112879 28398 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:34.140246 25705 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:34.115238 28298 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:33.960518 28337 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.180s
I20260709 12:18:33.965402 28369 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.160s
I20260709 12:18:34.107321 28371 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.178s
I20260709 12:18:34.155444 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:34.170262 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.156145 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.119614 28318 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.159135 25780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:34.174562 25780 raft_consensus.cc:606] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:34.174979 25780 raft_consensus.cc:614] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:34.175542 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:34.168267 28341 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.176308 28341 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.176535 28298 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.143846 28342 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:34.143441 25534 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:34.139101 28312 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:34.126744 28410 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:34.186076 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.179342 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:34.149466 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:34.181547 25750 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:34.157966 28091 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:34.157842 28343 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:34.260344 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 77651.2us
I20260709 12:18:34.248730 25708 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:34.300156 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 150764us
I20260709 12:18:34.314625 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:34.158332 28286 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:34.105273 28415 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.148303 25778 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:34.186815 25534 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:18:34.310110 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.330667 25534 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:34.321094 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:34.179744 28398 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:34.178458 28341 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:34.184510 28260 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.692s	user 0.007s	sys 0.004s
I20260709 12:18:34.179126 28345 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:34.178848 28342 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:34.341109 25638 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:34.168339 28376 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.154s
W20260709 12:18:34.343886 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 162444us
I20260709 12:18:34.344009 25603 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:34.305954 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 708) took 1212 ms. Trace:
I20260709 12:18:34.298856 28306 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.317489 28408 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:34.322474 28343 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:34.359189 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.320003 28286 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.317044 25676 leader_election.cc:400] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:34.323237 28301 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:34.141499 25532 rpcz_store.cc:269] 0709 12:18:16.650601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.681928 (+ 31327us) service_pool.cc:224] Handling call
0709 12:18:34.115288 (+17433360us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:27.959606 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:27.960381 (+   775us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:27.960447 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:18:27.960467 (+    20us) tablet.cc:662] Decoding operations
0709 12:18:27.961144 (+   677us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:27.961231 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:27.961259 (+    28us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:18:27.961749 (+   490us) tablet.cc:701] Row locks acquired
0709 12:18:27.961800 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:18:27.961994 (+   194us) write_op.cc:270] Start()
0709 12:18:27.962158 (+   164us) write_op.cc:276] Timestamp: P: 1783599507961920 usec, L: 0
0709 12:18:27.962181 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:18:27.963206 (+  1025us) log.cc:844] Serialized 1317 byte log entry
0709 12:18:28.622904 (+659698us) op_driver.cc:464] REPLICATION: finished
0709 12:18:28.695470 (+ 72566us) write_op.cc:301] APPLY: starting
0709 12:18:28.817022 (+121552us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:28.817184 (+   162us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:28.943272 (+126088us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:29.160592 (+217320us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:29.163205 (+  2613us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:18:29.163251 (+    46us) write_op.cc:312] APPLY: finished
0709 12:18:29.163580 (+   329us) log.cc:844] Serialized 35 byte log entry
0709 12:18:29.163784 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:29.163880 (+    96us) write_op.cc:454] Released schema lock
0709 12:18:29.163940 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:29.164408 (+   468us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:29.164437 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1490383,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":45889,"apply.run_cpu_time_us":8695,"apply.run_wall_time_us":479078,"num_ops":3,"prepare.queue_time_us":347887,"prepare.run_cpu_time_us":10568,"prepare.run_wall_time_us":313403,"raft.queue_time_us":210559,"raft.run_cpu_time_us":11669,"raft.run_wall_time_us":60382,"replication_time_us":660536,"thread_start_us":626577,"threads_started":4,"wal-append.queue_time_us":322783,"wal-append.run_cpu_time_us":38273,"wal-append.run_wall_time_us":1882957}]]}
I20260709 12:18:34.188648 28318 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.376778 25676 leader_election.cc:403] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:18:34.354305 28416 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.437s	user 0.001s	sys 0.000s
I20260709 12:18:34.347620 28287 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:34.323324 28312 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:18:34.347683 25711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a760df354a14284b004635d45d88a3e"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:18:34.378998 28286 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:34.395750 28273 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.376794 28345 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:34.397841 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:34.385025 28301 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.387367 28338 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:18:34.350970 28260 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.182478 26340 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:34.386765 28368 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
W20260709 12:18:34.409732 28418 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
W20260709 12:18:34.410499 28414 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:18:34.402580 28214 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:34.420224 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.390893 28306 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:34.386031 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 343096us
I20260709 12:18:34.408869 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.392577 28287 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:34.422845 25534 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
W20260709 12:18:34.412533 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 95711.9us
I20260709 12:18:34.484009 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.483227 25534 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 4, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:34.481045 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.510890 26336 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:18:34.511723 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 347648us
I20260709 12:18:34.512395 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:34.484874 25710 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:34.485343 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.419883 28139 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.489091 28318 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.422972 28408 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:34.492441 28376 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:34.492801 28350 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.699s	user 0.006s	sys 0.005s
I20260709 12:18:34.426395 28398 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:34.503151 28264 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.190s	user 0.012s	sys 0.004s
I20260709 12:18:34.401831 28210 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:18:34.357542 26337 rpcz_store.cc:276] 0709 12:18:33.093400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.611917 (+518517us) service_pool.cc:224] Handling call
0709 12:18:33.612234 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.617887 (+  5653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.617919 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:33.618500 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:18:33.779653 (+161153us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:33.786016 (+  6363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.800487 (+ 14471us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.180492 (+380005us) raft_consensus.cc:1685] finished
0709 12:18:34.184810 (+  4318us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.296736 (+111926us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31579,"raft.queue_time_us":64004,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":15579,"thread_start_us":117247,"threads_started":1,"wal-append.queue_time_us":177411}
I20260709 12:18:34.430796 28274 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
W20260709 12:18:34.584323 28340 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.761s	user 0.007s	sys 0.004s
I20260709 12:18:34.605474 28340 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.608023 28340 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.504244 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.614147 25712 raft_consensus.cc:2368] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:34.505064 28260 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:34.506778 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:34.509001 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:34.616245 26343 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:34.478845 25708 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:34.480167 25705 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
W20260709 12:18:34.479835 28024 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.000s
W20260709 12:18:34.557509 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45072.6us
W20260709 12:18:34.557648 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50483.9us
I20260709 12:18:34.515383 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:34.521857 26336 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:34.622944 25708 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:34.516247 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:34.623399 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:34.623850 25704 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:34.439620 28296 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.624722 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:34.624799 28296 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.628913 25708 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:34.507342 28287 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:34.512749 28391 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:18:34.589313 28264 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:34.548857 25634 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:34.515515 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.513247 25534 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:34.654794 25534 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:34.505766 28089 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:34.619603 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:34.487159 28306 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:34.514858 28139 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.627413 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:34.628404 25705 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:34.672066 25631 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:34.604022 28208 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.976s	user 0.010s	sys 0.002s
W20260709 12:18:34.603739 28428 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:18:34.615962 28186 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.508s	user 0.004s	sys 0.007s
I20260709 12:18:34.576298 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1941) took 1456 ms. Trace:
I20260709 12:18:34.687908 25637 rpcz_store.cc:276] 0709 12:18:33.119697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.122987 (+  3290us) service_pool.cc:224] Handling call
0709 12:18:33.123287 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.123602 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.123629 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:33.137005 (+ 13376us) log.cc:844] Serialized 26 byte log entry
0709 12:18:33.288395 (+151390us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b48002700d8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29ba8b8aa1 00007f29ba8afb37 00007f29ba8af9ca 00007f29ba9a726f 00007f29ba9a6f37 00007f29ba9bb55c 00007f29ba9bb508 00007f29ba9bb488 00007f29ba9bb443 00007f29ba9ba6ec 00007f29b98e4a96
0709 12:18:33.772922 (+484527us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b18004684c0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:18:33.886327 (+113405us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:33.886378 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.886502 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.576032 (+689530us) raft_consensus.cc:1685] finished
0709 12:18:34.576058 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.576232 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10510,"raft.run_cpu_time_us":2951,"raft.run_wall_time_us":141740,"spinlock_wait_cycles":15283840,"thread_start_us":113168,"threads_started":1,"wal-append.queue_time_us":1006985}
I20260709 12:18:34.580606 28408 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:34.579411 28350 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.586751 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:34.684746 25633 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:34.693567 25785 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:34.686317 25710 raft_consensus.cc:2376] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 7. Re-sending same reply.
W20260709 12:18:34.691583 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 307667us
I20260709 12:18:34.512040 28342 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.696727 25709 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:18:34.697458 25709 pending_rounds.cc:85] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 6
I20260709 12:18:34.697789 25709 pending_rounds.cc:107] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:18:34.698135 25709 raft_consensus.cc:2889] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:34.701503 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:34.700323 26307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.002s	sys 0.023s
W20260709 12:18:34.714433 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.678933 28306 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:34.431025 28410 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:34.692289 28258 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.767s	user 0.009s	sys 0.002s
W20260709 12:18:34.716239 26307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.002s	sys 0.023s
I20260709 12:18:34.725625 25704 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:34.627717 28024 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.774143 25708 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:34.775547 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 806786us
I20260709 12:18:34.648160 28213 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.648221 28286 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.786088 25631 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:34.786851 25637 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:34.576843 28343 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.576680 28345 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.801601 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:34.772636 28229 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.716374 28409 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 12:18:34.805279 25779 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:34.805634 25787 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:34.803373 25786 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:34.813638 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:34.807700 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 302277us
I20260709 12:18:34.690876 28208 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.818779 28229 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:34.806608 28429 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:18:34.784215 28258 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.827626 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:34.800251 28260 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.805994 28301 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:34.694799 28186 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.862890 25630 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:34.792277 28273 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.701395 28408 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.797619 28440 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:34.831187 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.862267 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:34.886914 25750 leader_election.cc:336] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.579s (SENT)
I20260709 12:18:34.864714 25676 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:34.824767 28441 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:18:34.871445 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 170035us
I20260709 12:18:34.845844 28298 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:34.916436 25711 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:34.917806 25711 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:34.836047 28415 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.867000 28301 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:34.943821 28447 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 12:18:34.944022 28444 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:18:34.913141 28089 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:34.862241 28260 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:34.923337 28440 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:34.945734 28286 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.929283 28001 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:34.946723 28267 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.027009 28267 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.028046 28446 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.915966 28318 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:18:35.030958 28462 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.031662 28462 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.019778 28229 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.982303 28368 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:18:35.020009 28186 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.925086 28274 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:18:34.980535 28296 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:34.979120 28264 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:35.022608 28350 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:35.050465 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 140555us
W20260709 12:18:35.033017 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 403) took 1603 ms (client timeout 2020 ms). Trace:
I20260709 12:18:34.946130 28298 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:34.976768 28301 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.021274 28440 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.064011 25706 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:35.051479 25785 rpcz_store.cc:269] 0709 12:18:33.430086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.580390 (+150304us) service_pool.cc:224] Handling call
0709 12:18:35.032803 (+1452413us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400258ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:35.032952 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2557678464}
I20260709 12:18:35.059013 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.059706 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.066103 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:35.068647 25705 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:35.043059 28368 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:35.023548 28089 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:18:35.099831 25709 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:35.102183 28451 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:35.103088 28451 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:35.103690 28465 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.104242 28451 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.028788 28446 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:35.106715 28460 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:18:35.108172 28301 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:35.109035 28301 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.109517 28301 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.110939 28267 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.050689 28318 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:35.105201 27952 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.370s	user 0.006s	sys 0.007s
I20260709 12:18:35.043382 28001 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.073199 28139 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.070967 28213 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.072985 28186 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.123330 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:35.118019 28312 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:18:35.106722 28208 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:35.169628 28450 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 12:18:35.169991 28220 consensus_meta.cc:287] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.257s	user 0.010s	sys 0.004s
I20260709 12:18:35.076617 28398 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:18:35.170608 28220 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.096225 28350 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:18:35.122781 28318 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.104300 28465 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.176975 28465 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.076778 28440 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.119380 28310 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:18:35.178730 28462 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.144582 28186 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.123648 27952 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.211844 28186 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.212081 25784 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:35.202668 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.189497 28258 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.215376 28440 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.187686 28408 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:35.180994 28001 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:35.188194 28260 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.211849 27952 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.239280 28446 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:35.239883 28287 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
I20260709 12:18:35.240375 28287 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.308779 28220 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.223922 28306 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.308884 28301 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.342087 28398 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:35.321507 27534 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.339921 27952 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.384697 28267 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.392387 28398 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.400979 28267 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.413995 28398 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.415169 28267 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.394759 27952 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.423269 27952 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.429462 28287 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:35.430936 28306 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:35.484187 25604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.010s	sys 0.018s
W20260709 12:18:35.485337 25604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.011s	sys 0.018s
W20260709 12:18:35.487399 25601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.000s	sys 0.014s
W20260709 12:18:35.488206 25601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.000s	sys 0.014s
W20260709 12:18:35.487262 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.020s	sys 0.041s
W20260709 12:18:35.494352 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.021s	sys 0.042s
I20260709 12:18:35.494167 25638 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:35.494877 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 584130us
I20260709 12:18:35.497432 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:35.498510 25679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.009s	sys 0.017s
W20260709 12:18:35.499202 25679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.009s	sys 0.018s
I20260709 12:18:35.508868 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:35.507313 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 882597us
I20260709 12:18:35.510943 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:35.511088 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:35.512135 25634 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:35.514041 25637 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:35.510028 25631 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:35.512013 25633 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:35.523075 25638 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:35.525766 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:35.528417 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:35.117081 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 216736us
I20260709 12:18:35.556499 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.558220 25709 raft_consensus.cc:2410] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:35.559281 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:35.574548 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.576397 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:35.581164 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:35.585772 26307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.004s	sys 0.036s
W20260709 12:18:35.624617 26307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.004s	sys 0.036s
I20260709 12:18:35.625005 25709 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:35.627529 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.627640 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.628892 26338 raft_consensus.cc:2410] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:35.393862 27534 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.630429 26336 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:35.630941 26341 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:35.630688 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:35.673420 25534 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.017s	sys 0.006s
W20260709 12:18:35.674273 25534 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.018s	sys 0.006s
I20260709 12:18:35.682055 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.682380 27534 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.679795 25631 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:35.680641 25634 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:35.740289 25633 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:18:35.741683 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 88140.7us
I20260709 12:18:35.745652 26337 raft_consensus.cc:2368] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4.
I20260709 12:18:35.683362 26339 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:35.750402 26338 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:35.751595 28408 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:35.752161 28408 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.752521 28408 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:35.754439 28342 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.004s	sys 0.009s
W20260709 12:18:35.754971 28342 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.004s	sys 0.009s
I20260709 12:18:35.755329 28287 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.754999 28426 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:35.756145 28287 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.757761 28426 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:35.758736 28229 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.759701 28301 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.760673 28301 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.762907 28260 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.763521 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.763901 28260 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.764636 28451 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:35.765573 28451 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.762919 28457 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:18:35.769372 28461 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:35.770861 28470 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.771059 28459 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:35.771739 28470 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.772275 28274 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.773119 28274 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.774370 28480 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.774614 28446 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:35.775323 26341 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:35.775287 28480 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.775826 28446 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:35.759510 28229 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.778036 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:35.799556 28306 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.747309 28296 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:18:35.800447 28306 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:35.803980 28417 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.695s	user 0.008s	sys 0.012s
I20260709 12:18:35.806461 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2160) took 2174 ms. Trace:
I20260709 12:18:35.807072 25713 rpcz_store.cc:276] 0709 12:18:33.632333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.633936 (+  1603us) service_pool.cc:224] Handling call
0709 12:18:33.634364 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.719249 (+ 84885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.719293 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:33.719840 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:18:33.833535 (+113695us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:33.833572 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.833633 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.802718 (+1969085us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:35.805761 (+  3043us) raft_consensus.cc:1685] finished
0709 12:18:35.805795 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.806217 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98795,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":20139,"spinlock_wait_cycles":11877632,"thread_start_us":133088,"threads_started":2,"wal-append.queue_time_us":181421}
I20260709 12:18:35.807195 28480 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.816061 28274 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.809973 28398 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.831902 28398 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:35.835361 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61144us
I20260709 12:18:35.840821 28274 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.843464 26336 raft_consensus.cc:2368] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:35.842103 28274 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:35.856866 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45956.8us
I20260709 12:18:35.771773 28457 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:35.861673 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.868739 28260 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:35.873872 25709 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:35.875260 28301 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:35.875636 25711 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.957s	user 0.010s	sys 0.009s
I20260709 12:18:35.876245 25711 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.856225 28287 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.878923 28457 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.879436 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.855899 28296 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.880903 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:35.881390 28296 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:35.882066 28267 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:35.883315 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.884361 26341 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:35.885365 28470 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.876703 28306 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.886133 28342 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:35.884599 28296 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:35.888473 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.881407 28457 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:35.889068 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:35.889748 28457 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.891546 28274 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:35.843124 28229 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.891057 28457 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.895592 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.896564 25713 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:35.897790 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.784267 28426 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:35.900671 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:35.901512 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:35.903371 25713 raft_consensus.cc:2368] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:35.903903 28398 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:35.905141 28479 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:35.906101 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.907099 28397 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:35.897547 25706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.006s	sys 0.014s
I20260709 12:18:35.913314 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2098) took 1413 ms. Trace:
I20260709 12:18:35.910712 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:35.914930 25710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2100) took 1414 ms (client timeout 1873 ms). Trace:
W20260709 12:18:35.915473 25710 rpcz_store.cc:269] 0709 12:18:34.500088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.515110 (+ 15022us) service_pool.cc:224] Handling call
0709 12:18:35.914513 (+1399403us) spinlock_profiling.cc:243] Waited 63.5 ms on lock 0x7b5400104354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:35.914715 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145960704}
I20260709 12:18:35.902387 28426 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:35.913941 25706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.007s	sys 0.014s
I20260709 12:18:35.917212 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.916088 25709 raft_consensus.cc:2376] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8. Re-sending same reply.
I20260709 12:18:35.912050 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:35.921787 28477 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:35.913764 25708 rpcz_store.cc:276] 0709 12:18:34.499478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.630255 (+130777us) service_pool.cc:224] Handling call
0709 12:18:34.773936 (+143681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.776333 (+  2397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.912869 (+1136536us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002ec7d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:35.913108 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329634048}
I20260709 12:18:35.944060 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:35.955391 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.936300 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.962373 25706 raft_consensus.cc:2410] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:18:35.900799 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:35.968119 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:35.970786 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.971526 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.963063 28229 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:36.077436 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:36.232313 28310 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:36.045938 28470 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.043783 28405 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.203948 28298 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.221196 28318 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.105094 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:35.984587 26341 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:35.979704 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:35.985273 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:36.003633 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 93477.9us
I20260709 12:18:36.037940 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:36.076615 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:36.187927 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:36.365630 25632 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:36.100975 25708 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:36.105315 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:36.352262 25711 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.046262 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:36.364177 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:36.462123 25713 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:36.361178 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:36.363757 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:36.143275 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:36.009055 25750 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:36.043505 25603 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:36.340025 25677 connection.cc:256] RPC call timeout handler was delayed by -1.0756630539999605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
W20260709 12:18:36.203796 28498 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:36.341768 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2175) took 1268 ms. Trace:
I20260709 12:18:36.487383 25710 raft_consensus.cc:2410] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:18:36.308040 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1554) took 2915 ms. Trace:
I20260709 12:18:36.084627 25787 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:36.018608 27952 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:36.186640 28267 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.082851 27923 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.498136 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:36.217578 25630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1590) took 2837 ms (client timeout 2095 ms). Trace:
I20260709 12:18:35.993556 28451 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.366360 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:36.191620 28024 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.367630 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:36.366921 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:36.396695 28139 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.359822 28273 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.358793 28369 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:36.566354 25782 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:36.187002 28264 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.312124 26337 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:36.581573 25631 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:36.186910 28371 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.996227 28457 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.496541 25784 rpcz_store.cc:276] 0709 12:18:33.392475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.740653 (+348178us) service_pool.cc:224] Handling call
0709 12:18:33.740957 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.212473 (+1471516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.307320 (+1094847us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54000ed654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:36.307783 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3382796544}
I20260709 12:18:36.399243 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:36.496047 25705 rpcz_store.cc:276] 0709 12:18:35.073318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.073423 (+   105us) service_pool.cc:224] Handling call
0709 12:18:35.073698 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.101035 (+ 27337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.101069 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:35.101117 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.101154 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.341152 (+1239998us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b5400147154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:36.341231 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.341546 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62403968}
I20260709 12:18:36.340384 28342 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:36.523401 25630 rpcz_store.cc:269] 0709 12:18:33.379858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.380069 (+   211us) service_pool.cc:224] Handling call
0709 12:18:36.217025 (+2836956us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003011d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:36.217312 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3365334784}
I20260709 12:18:36.075253 28346 raft_consensus.cc:993] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:18:36.530555 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:36.587608 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:36.323858 26340 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:36.187099 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:36.411247 28442 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 12:18:36.533329 28267 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.013373 28214 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:36.534359 25783 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 2.
I20260709 12:18:36.285640 28229 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.535327 28213 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:36.386689 28368 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:36.536127 27923 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.537336 27952 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:36.537755 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 558889us
I20260709 12:18:36.538463 28420 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:36.539242 28440 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.539548 25710 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:18:36.540815 28426 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:36.541512 27936 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:36.013350 28461 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:36.542850 28411 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.543144 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 496973us
I20260709 12:18:36.543964 28441 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:36.545032 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:36.544915 28341 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:36.545879 28490 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:36.546452 27955 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:36.547457 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:36.548273 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:36.550381 25635 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:36.324548 28370 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:36.554807 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:36.637279 25750 leader_election.cc:400] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:36.530535 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:36.637733 25750 leader_election.cc:403] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:18:36.395565 28338 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:36.558391 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 537237us
I20260709 12:18:36.462308 28310 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:36.512720 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:36.411375 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:36.084290 28287 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.649891 28290 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:36.651222 28210 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:36.550540 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2049) took 2592 ms. Trace:
I20260709 12:18:36.374940 28398 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.155s
I20260709 12:18:36.583806 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:57966 (request call id 2092) took 2164 ms. Trace:
I20260709 12:18:36.585160 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:36.554857 28489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.016s	sys 0.031s
I20260709 12:18:36.588229 25713 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:36.636993 28370 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:36.579523 28273 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.596684 28342 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.597074 28346 raft_consensus.cc:1081] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:18:36.598711 28422 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:36.599323 28481 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:18:36.600919 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 239905us
I20260709 12:18:36.410351 28403 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.173s
I20260709 12:18:36.673252 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:36.566815 28369 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:36.615525 25709 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:36.618549 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2180) took 1108 ms. Trace:
I20260709 12:18:36.688210 25710 rpcz_store.cc:276] 0709 12:18:35.504045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.538577 (+1034532us) service_pool.cc:224] Handling call
0709 12:18:36.539042 (+   465us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:36.612319 (+ 73277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.612599 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.580005 28318 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.620141 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:36.692088 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2103) took 1115 ms. Trace:
I20260709 12:18:36.692377 25783 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:36.693838 25779 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:36.613840 27952 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:36.622444 28457 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.622800 28440 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.568384 28470 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.358031 28427 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:36.605183 28368 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:36.567211 28405 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.642025 28337 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:18:36.642356 28298 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.643735 25706 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:36.643954 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:36.623996 27936 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.645304 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:36.645262 28426 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.646263 28451 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:18:36.720417 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2107) took 1099 ms. Trace:
I20260709 12:18:36.648458 28310 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.209219 28408 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.674230 28461 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.655664 25711 rpcz_store.cc:276] 0709 12:18:34.347361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.347527 (+   166us) service_pool.cc:224] Handling call
0709 12:18:36.338341 (+1990814us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54003aa554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5986a3d 00007f29ba956cd0 00007f29b3edb254 00007f29b3edb1e2 00007f29b3edb111 00007f29b3edb08c 00007f29b3eda342 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:36.511565 (+173224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206860544}
I20260709 12:18:36.668581 28441 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:36.660091 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 115253us
I20260709 12:18:36.594657 28371 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.655804 25712 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:36.658310 25708 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:36.650908 28287 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.640055 28338 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:36.668424 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 257306us
W20260709 12:18:36.666891 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 150896us
I20260709 12:18:36.652679 28024 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:36.661551 28489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.017s	sys 0.031s
I20260709 12:18:36.654387 28210 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:36.638696 28493 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:36.628213 28139 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.631302 28214 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.651876 28213 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.603924 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:36.624053 28264 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.691186 25752 leader_election.cc:400] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:36.774122 25752 leader_election.cc:403] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:18:36.692646 25709 rpcz_store.cc:276] 0709 12:18:35.573084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.551871 (+978787us) service_pool.cc:224] Handling call
0709 12:18:36.570618 (+ 18747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.687717 (+117099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.687958 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.702140 25785 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:36.754748 28267 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:36.708643 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:36.703397 28274 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:36.717244 25751 connection.cc:256] RPC call timeout handler was delayed by -1.0488115779999134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
I20260709 12:18:36.721027 25706 rpcz_store.cc:276] 0709 12:18:35.620757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.602981 (+982224us) service_pool.cc:224] Handling call
0709 12:18:36.625081 (+ 22100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.720064 (+ 94983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.720340 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.722002 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2102) took 2219 ms. Trace:
I20260709 12:18:36.593842 25637 raft_consensus.cc:2410] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:36.656757 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:36.725343 28345 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:36.734583 28347 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:18:36.735075 28368 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.735359 28273 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.735505 28370 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.738067 28405 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:36.719841 27936 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:36.740113 28440 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.740875 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:36.742457 28369 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.829138 28369 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:36.829602 28310 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:36.830899 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:36.831897 28290 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:36.832302 28290 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:36.833041 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:36.834000 28024 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:36.834669 28461 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.839043 27923 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.839368 28405 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:36.839208 27936 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.839850 28370 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:36.840102 28405 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:36.840548 25713 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:36.841509 27936 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:36.841189 28405 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.843122 28024 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.745350 28340 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:36.843775 28024 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.846436 28024 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:36.848735 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2109) took 1089 ms. Trace:
I20260709 12:18:36.849148 25710 rpcz_store.cc:276] 0709 12:18:35.759636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.699413 (+939777us) service_pool.cc:224] Handling call
0709 12:18:36.709320 (+  9907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.763644 (+ 54324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.763691 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.775676 (+ 11985us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:36.775742 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.848487 (+ 72745us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b54002238d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:36.848537 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.848672 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96002944}
W20260709 12:18:36.746652 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 107623us
I20260709 12:18:36.722050 28432 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:36.850610 28483 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:36.676208 28490 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:36.756090 25711 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:36.851687 28490 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:36.852784 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2186) took 1055 ms. Trace:
I20260709 12:18:36.852738 28490 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.690742 28411 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.853241 25711 rpcz_store.cc:276] 0709 12:18:35.797518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.733607 (+936089us) service_pool.cc:224] Handling call
0709 12:18:36.751510 (+ 17903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.851615 (+100105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.852599 (+   984us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5400147b50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:36.852731 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40367488}
I20260709 12:18:36.675706 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:36.855988 28411 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.856765 28441 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.857029 25782 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:36.723877 28258 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:36.858629 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:36.858721 28258 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:36.859287 26337 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:36.861372 28507 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.266s	user 0.004s	sys 0.000s
I20260709 12:18:36.654663 25781 rpcz_store.cc:276] 0709 12:18:33.957711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.974406 (+ 16695us) service_pool.cc:224] Handling call
0709 12:18:34.056646 (+ 82240us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:34.120119 (+ 63473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.120175 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:34.141645 (+ 21470us) log.cc:844] Serialized 590 byte log entry
0709 12:18:35.847734 (+1706089us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b18003895c0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:18:36.017913 (+170179us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:36.017962 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.018112 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.550055 (+531943us) raft_consensus.cc:1685] finished
0709 12:18:36.550076 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.550332 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92687,"raft.queue_time_us":66780,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":12483,"spinlock_wait_cycles":51781376,"thread_start_us":81456,"threads_started":1,"wal-append.queue_time_us":2405889}
I20260709 12:18:36.722690 28457 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.791850 25712 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:36.800834 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 157006us
I20260709 12:18:36.720500 27955 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.801245 28210 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.807385 28274 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.806826 28345 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.812012 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:36.779888 25707 rpcz_store.cc:276] 0709 12:18:34.500612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.676552 (+175940us) service_pool.cc:224] Handling call
0709 12:18:34.689562 (+ 13010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.689836 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:36.328061 (+1638225us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b180030b440. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b289f455 00007f29b93873c7 00007f29b59999a7 00007f29b59a90bf
0709 12:18:36.328387 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.688260 (+359873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.693166 (+  4906us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:36.698701 (+  5535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.703132 (+  4431us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.719799 (+ 16667us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9095,"prepare.queue_time_us":239993,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":2153,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4490,"spinlock_wait_cycles":42154368,"thread_start_us":497717,"threads_started":2,"wal-append.queue_time_us":450802}
I20260709 12:18:36.818028 28139 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.818884 25784 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:36.819047 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2172) took 1759 ms. Trace:
I20260709 12:18:36.819530 28089 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:36.819824 28343 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.042s	sys 0.052s
I20260709 12:18:36.820588 28264 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:36.821949 28426 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.822783 28368 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:36.823777 28451 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:36.825630 25780 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:36.827958 28214 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.828351 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:36.818071 28234 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.851061 28432 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:36.773447 25786 raft_consensus.cc:2410] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:36.899853 28432 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.864434 28290 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.903630 28290 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:36.907279 28234 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:36.909525 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.909614 28234 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.716883 28318 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.912159 28318 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.912746 28422 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:36.913184 28422 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:36.914808 28346 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:36.915231 28346 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:36.916199 28346 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.917500 28346 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:36.865628 25710 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:36.921051 28342 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:36.873008 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2187) took 1055 ms. Trace:
I20260709 12:18:36.809969 28337 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:36.878330 28091 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:36.922796 28337 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:36.923719 28457 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:36.880087 27955 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.880429 28345 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.924949 28337 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:36.880949 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 114239us
I20260709 12:18:36.922046 25712 rpcz_store.cc:276] 0709 12:18:35.817474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.759307 (+941833us) service_pool.cc:224] Handling call
0709 12:18:36.779208 (+ 19901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.872710 (+ 93502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.872945 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.926152 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:36.928485 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:36.930869 28274 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.881704 28024 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:36.881867 28470 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.872763 28441 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 12:18:36.882726 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 212750us
I20260709 12:18:36.931705 28024 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:36.876466 25704 rpcz_store.cc:276] 0709 12:18:35.059455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.067262 (+  7807us) service_pool.cc:224] Handling call
0709 12:18:35.067618 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.067870 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:35.116752 (+ 48882us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.401739 (+1284987us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b1800415360. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a 00007f29b5999a2a
0709 12:18:36.772040 (+370301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.790047 (+ 18007us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:36.803949 (+ 13902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.818715 (+ 14766us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.818865 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21121,"prepare.queue_time_us":47202,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":5692,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":18383,"spinlock_wait_cycles":6410496,"thread_start_us":349550,"threads_started":1}
I20260709 12:18:36.932935 28024 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.858446 28213 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.932361 28442 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:36.934837 28510 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:18:36.935860 28442 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:36.937208 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:36.776006 28298 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.839144 28371 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:36.891378 28343 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.042s	sys 0.052s
I20260709 12:18:36.929397 28267 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:36.930603 28089 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.933383 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:36.839654 28287 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:36.881079 28210 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:36.868265 25778 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:36.892936 28229 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:36.883489 28340 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:36.939597 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:36.889753 25631 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:36.945888 25779 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:36.947774 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:36.954629 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:36.968205 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:36.937522 28501 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:18:36.966908 26344 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:36.963217 25783 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:36.966884 28264 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:36.982630 25752 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:36.995855 25785 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:36.970309 28420 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:36.997153 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:36.997889 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:36.999644 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 197959us
I20260709 12:18:36.971369 28267 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:36.938539 28338 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.971763 28403 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:36.966233 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.003778 26336 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:37.004691 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:36.994947 28264 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.010579 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:36.965667 27952 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:36.975401 28340 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:37.000156 28287 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:37.018564 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:37.036463 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:37.037202 26344 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:37.014765 28264 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:37.039319 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.047125 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:37.043052 26339 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:37.016709 28459 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:37.021507 28287 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.077248 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:37.084311 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44269.8us
I20260709 12:18:37.039284 27936 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:37.083966 28493 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:37.083767 28287 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:37.096235 28493 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.093312 27936 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.098978 27936 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:37.101475 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.084550 28343 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:37.103279 28343 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.104698 28483 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:37.105165 28483 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.105535 28483 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:37.077157 28459 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:37.094125 28369 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.107148 28459 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:37.107419 28369 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:37.079370 28229 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:37.108918 28229 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:37.110368 27534 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:37.111232 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.111855 27534 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:37.113412 28301 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:37.113798 28301 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:37.116892 26343 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:37.077291 28481 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:37.016767 28457 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
W20260709 12:18:37.115975 28505 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.214s	user 0.000s	sys 0.003s
I20260709 12:18:37.077993 26335 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:37.136309 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.089951 28371 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:37.150856 28403 raft_consensus.cc:670] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 12:18:37.151638 28403 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:37.152139 28403 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:37.153046 28403 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.153404 28258 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.154953 28481 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.144367 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.145008 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.158639 25631 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:37.145606 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.150655 26339 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:18:37.146663 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 450304us
I20260709 12:18:37.163756 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.095702 28267 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.168654 26339 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:37.169019 28340 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:37.169620 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.170696 28298 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:37.170887 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.173130 26339 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:37.160033 28371 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:37.150208 28089 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:37.189713 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45160.5us
W20260709 12:18:37.193401 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.205207 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 352029us
I20260709 12:18:37.211879 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.211879 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.216146 26336 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:37.164353 28408 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:37.226015 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:37.200147 28371 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:37.166170 28493 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:37.190477 28139 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:18:37.233673 28369 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:37.270077 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:37.181296 28301 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.259415 28258 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:37.302668 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 178610us
W20260709 12:18:37.309273 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.310192 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 209584us
I20260709 12:18:37.269165 28229 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:37.270778 27952 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:37.315500 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 103074us
I20260709 12:18:37.350086 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:37.342752 28427 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.010s
I20260709 12:18:37.317121 26335 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:37.356837 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:37.354511 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.381531 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:37.357506 26340 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.746s	user 0.004s	sys 0.008s
I20260709 12:18:37.348675 28345 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:37.380081 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.398242 25783 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:18:37.384505 28301 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:37.402242 25783 pending_rounds.cc:85] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 2
I20260709 12:18:37.402684 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:37.402285 25632 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:37.406665 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:37.400470 28370 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.004s
I20260709 12:18:37.426882 25707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:18:37.427824 25707 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:37.428186 25707 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:37.406649 25783 pending_rounds.cc:107] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:18:37.359509 26337 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:37.379995 28343 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:37.342361 28519 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:37.384563 28441 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:18:37.359426 28517 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:18:37.361160 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:37.429298 25706 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:37.519387 25782 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:37.467381 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.682803 25632 raft_consensus.cc:2410] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:37.440320 25710 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:37.683840 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.476608 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.686285 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:37.425838 28310 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.005s
I20260709 12:18:37.687593 25783 raft_consensus.cc:2889] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:37.507181 28504 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 12:18:37.687834 25637 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:37.403141 28089 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.439494 28345 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:37.398310 28210 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.450s
I20260709 12:18:37.679558 25712 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:37.695063 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:38.543215 28537 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 12:18:37.708786 28508 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.494s	user 0.000s	sys 0.001s
I20260709 12:18:37.856549 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:37.863893 25704 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:37.470019 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:38.546985 25705 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
W20260709 12:18:38.544435 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3749527860002217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:18:38.049448 26344 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.010s	user 0.006s	sys 0.005s
I20260709 12:18:37.891067 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:37.465858 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:37.471144 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:38.548764 26344 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:18:37.987293 25634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.004s	sys 0.016s
I20260709 12:18:38.053556 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:38.604710 25634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.007s	sys 0.034s
W20260709 12:18:37.694564 26339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.009s	sys 0.007s
I20260709 12:18:37.869220 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:38.607884 26339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.434s	user 0.022s	sys 0.032s
I20260709 12:18:38.608144 25781 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:38.547628 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2190) took 1120 ms. Trace:
I20260709 12:18:37.933043 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:37.445676 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.934651 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:38.547694 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.430126 25636 raft_consensus.cc:2410] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:38.548045 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2197) took 1118 ms. Trace:
I20260709 12:18:38.609560 25705 rpcz_store.cc:276] 0709 12:18:37.427551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.441041 (+ 13490us) service_pool.cc:224] Handling call
0709 12:18:38.547566 (+1106525us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.611828 25704 rpcz_store.cc:276] 0709 12:18:37.429565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.659328 (+229763us) service_pool.cc:224] Handling call
0709 12:18:37.855259 (+195931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:38.546023 (+690764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.547989 (+  1966us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:38.002274 25707 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.001s
I20260709 12:18:38.585551 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 428) took 2331 ms. Trace:
W20260709 12:18:38.585868 25784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.002s	sys 0.009s
I20260709 12:18:38.613009 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:38.613349 25784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.003s	sys 0.009s
I20260709 12:18:38.614207 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1569) took 1230 ms. Trace:
I20260709 12:18:37.703016 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.020358 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:38.537524 28528 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:18:38.614995 25783 rpcz_store.cc:276] 0709 12:18:37.354907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.355077 (+   170us) service_pool.cc:224] Handling call
0709 12:18:37.355469 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:38.584505 (+1229036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.584836 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.636592 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2191) took 1208 ms. Trace:
W20260709 12:18:38.355748 28483 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.250s	user 0.006s	sys 0.010s
I20260709 12:18:37.894697 28517 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:18:38.296165 28549 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:18:38.635131 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1615) took 1315 ms. Trace:
W20260709 12:18:37.893261 28422 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.980s	user 0.013s	sys 0.000s
W20260709 12:18:38.285334 28459 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.178s	user 0.009s	sys 0.003s
W20260709 12:18:38.109601 28368 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.167s	user 0.013s	sys 0.000s
W20260709 12:18:38.157799 28451 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.211s	user 0.008s	sys 0.004s
W20260709 12:18:38.647192 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 1.19101e+06us
W20260709 12:18:38.632354 26344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 663) took 1663 ms (client timeout 1951 ms). Trace:
I20260709 12:18:38.675936 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.636345 25638 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:18:38.637338 25713 rpcz_store.cc:276] 0709 12:18:37.427883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.440903 (+ 13020us) service_pool.cc:224] Handling call
0709 12:18:38.636527 (+1195624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.613751 25705 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
W20260709 12:18:37.516754 28290 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.007s
I20260709 12:18:37.518394 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2060) took 1443 ms. Trace:
I20260709 12:18:37.446739 26340 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:38.638628 25632 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:37.857926 28398 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:37.894191 28534 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
W20260709 12:18:38.289757 27936 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.190s	user 0.018s	sys 0.000s
W20260709 12:18:37.932143 28523 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.547s	user 0.001s	sys 0.001s
W20260709 12:18:37.945269 28481 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.009s	sys 0.026s
W20260709 12:18:37.928745 28541 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:37.868432 28519 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:38.613029 25707 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:38.639238 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:38.205168 28427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.016s	sys 0.001s
I20260709 12:18:38.642589 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2116) took 2393 ms. Trace:
I20260709 12:18:38.642925 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:38.616148 28493 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.013s	sys 0.022s
W20260709 12:18:38.113205 28544 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 12:18:38.470263 28287 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.374s	user 0.014s	sys 0.000s
I20260709 12:18:38.613255 25787 rpcz_store.cc:276] 0709 12:18:36.213904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.863487 (+649583us) service_pool.cc:224] Handling call
0709 12:18:36.880192 (+ 16705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.880564 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.880594 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.881092 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.250632 (+369540us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:37.250682 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.250773 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.545350 (+1294577us) raft_consensus.cc:1685] finished
0709 12:18:38.545383 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.545571 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33889,"thread_start_us":369062,"threads_started":1,"wal-append.queue_time_us":933081}
I20260709 12:18:38.585044 28337 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:38.383181 28301 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.968s	user 0.007s	sys 0.005s
I20260709 12:18:38.641366 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1572) took 1186 ms. Trace:
W20260709 12:18:37.868861 28264 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.782s	user 0.003s	sys 0.007s
I20260709 12:18:38.645493 25636 rpcz_store.cc:276] 0709 12:18:37.319238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.319530 (+   292us) service_pool.cc:224] Handling call
0709 12:18:38.635030 (+1315500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.677439 25629 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
W20260709 12:18:38.676465 26344 rpcz_store.cc:269] 0709 12:18:36.941766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.013586 (+ 71820us) service_pool.cc:224] Handling call
0709 12:18:38.605012 (+1591426us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.676712 25783 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:38.676667 28459 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:38.736279 25780 rpcz_store.cc:276] 0709 12:18:37.455431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.455647 (+   216us) service_pool.cc:224] Handling call
0709 12:18:38.641298 (+1185651us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.646286 28419 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:38.644234 28139 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:38.681147 28451 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:38.702463 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 647) took 5443 ms. Trace:
I20260709 12:18:38.704003 28519 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:38.639333 28089 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:38.721570 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 654) took 1829 ms. Trace:
I20260709 12:18:38.707301 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 658) took 1759 ms. Trace:
W20260709 12:18:38.755048 28554 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:18:38.704213 28371 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.433s	user 0.007s	sys 0.005s
W20260709 12:18:38.701205 28481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.541s	user 0.018s	sys 0.043s
I20260709 12:18:38.680938 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:38.710976 28427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.019s	sys 0.019s
I20260709 12:18:38.685472 28415 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.754844 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.755697 25707 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:38.645898 28483 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:38.723647 28493 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.264s	user 0.013s	sys 0.023s
I20260709 12:18:38.680787 28422 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:38.704610 25709 rpcz_store.cc:276] 0709 12:18:36.249431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.802116 (+552685us) service_pool.cc:224] Handling call
0709 12:18:36.868099 (+ 65983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.918962 (+ 50863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.918992 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.919530 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.289408 (+369878us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:37.289463 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.289554 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.642325 (+1352771us) raft_consensus.cc:1685] finished
0709 12:18:38.642350 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.642518 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2755,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":369422,"threads_started":1,"wal-append.queue_time_us":1061987}
I20260709 12:18:38.757431 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.699354 27936 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:38.756911 26340 rpcz_store.cc:276] 0709 12:18:33.247488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.810518 (+563030us) service_pool.cc:224] Handling call
0709 12:18:33.810830 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.945734 (+134904us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b54006cb720. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5906dc0 00007f29b59911c0 00007f29b5992231 00007f29b598ed8f 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c
0709 12:18:36.322892 (+2377158us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b54006cb720. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5904a1d 00007f29b58eb1a4 00007f29b596d447 00007f29b5985fe7 00007f29b59877de 00007f29b5991b49 00007f29b5992416 00007f29b598ed8f 00007f29b598e83e 00007f29ba953419
0709 12:18:38.684414 (+2361522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.690445 (+  6031us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57975936}
I20260709 12:18:38.754832 28264 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:38.704610 28346 consensus_meta.cc:287] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.787s	user 0.010s	sys 0.005s
W20260709 12:18:38.703127 28258 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.389s	user 0.006s	sys 0.006s
I20260709 12:18:38.680608 25778 rpcz_store.cc:276] 0709 12:18:36.075048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.109001 (+ 33953us) service_pool.cc:224] Handling call
0709 12:18:37.518099 (+1409098us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000b04d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:37.518292 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398925184}
W20260709 12:18:38.758023 28552 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:18:38.766023 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:38.733184 28287 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:38.770849 25706 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:38.762352 26341 rpcz_store.cc:276] 0709 12:18:36.940188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.947852 (+  7664us) service_pool.cc:224] Handling call
0709 12:18:36.952901 (+  5049us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.217338 (+264437us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:37.405977 (+188639us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.694748 (+288771us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:37.849483 (+154735us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.849651 (+   168us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:37.849708 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.849799 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.691550 (+841751us) raft_consensus.cc:1685] finished
0709 12:18:38.691574 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.699728 (+  8154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9392,"prepare.queue_time_us":778509,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":56081,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":121369,"thread_start_us":608955,"threads_started":3,"wal-append.queue_time_us":491770}
W20260709 12:18:38.772734 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 91903.7us
I20260709 12:18:38.775805 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:57966 (request call id 2125) took 1354 ms. Trace:
I20260709 12:18:38.735270 28337 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:38.776976 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.777113 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.786820 25707 rpcz_store.cc:276] 0709 12:18:37.417912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.418115 (+   203us) service_pool.cc:224] Handling call
0709 12:18:38.771684 (+1353569us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":209}
I20260709 12:18:38.787192 25783 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:38.787484 25780 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:38.760083 28139 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:38.772670 28517 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:38.768400 28345 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:38.790283 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.791381 28337 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:38.777401 28415 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:38.791481 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:38.795564 26340 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:38.767832 28371 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:38.799062 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 670) took 1582 ms. Trace:
I20260709 12:18:38.799393 26340 rpcz_store.cc:276] 0709 12:18:37.216909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.795131 (+1578222us) service_pool.cc:224] Handling call
0709 12:18:38.795358 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:38.796530 (+  1172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.799002 (+  2472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.802697 25787 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:38.803366 25779 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:38.797626 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:38.797974 25636 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:38.796600 28459 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:38.807773 28264 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:38.808550 28287 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:38.794979 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.809532 25785 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:37.526328 26335 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:38.814451 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:38.815456 28196 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:18:38.818902 26336 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:38.819733 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.821199 27936 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:38.812881 28427 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:38.811050 28519 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:38.823225 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:38.825496 28258 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:38.826004 25781 raft_consensus.cc:2410] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:38.825546 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:38.934412 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 108957us
I20260709 12:18:38.826853 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:38.829301 28547 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:18:38.938505 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1575) took 1076 ms. Trace:
I20260709 12:18:38.939069 25781 rpcz_store.cc:276] 0709 12:18:37.862388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.868910 (+  6522us) service_pool.cc:224] Handling call
0709 12:18:38.938399 (+1069489us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:38.829881 28371 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:38.914734 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1619) took 1543 ms. Trace:
I20260709 12:18:38.816046 25778 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:18:38.922755 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.923566 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1614) took 1595 ms. Trace:
I20260709 12:18:38.927542 25780 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:38.941017 25633 rpcz_store.cc:276] 0709 12:18:37.360034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.360280 (+   246us) service_pool.cc:224] Handling call
0709 12:18:37.401987 (+ 41707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.683158 (+281171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.683186 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:37.683235 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:37.683264 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.903478 (+1220214us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400371ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:38.903539 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.903736 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646068480}
I20260709 12:18:38.944367 25631 rpcz_store.cc:276] 0709 12:18:37.318866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.347558 (+ 28692us) service_pool.cc:224] Handling call
0709 12:18:37.360518 (+ 12960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.407280 (+ 46762us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:38.550731 (+1143451us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.913296 (+362565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:38.913384 (+    88us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:38.913453 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.913553 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.913728 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":107911,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":9163,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":2479,"thread_start_us":462395,"threads_started":2,"wal-append.queue_time_us":317232}
W20260709 12:18:38.953189 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.955544 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.931092 28427 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:38.980031 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1568) took 1636 ms (client timeout 2066 ms). Trace:
W20260709 12:18:38.980532 25785 rpcz_store.cc:269] 0709 12:18:37.344023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.349789 (+  5766us) service_pool.cc:224] Handling call
0709 12:18:38.979792 (+1630003us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54001ddb54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:38.979948 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3354271104}
I20260709 12:18:38.981693 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.947449 28551 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:38.982532 28551 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.982848 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 736) took 2033 ms. Trace:
W20260709 12:18:38.980086 28546 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:18:38.985026 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:38.986377 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:38.968567 28517 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:38.988956 28517 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:38.983306 26339 rpcz_store.cc:276] 0709 12:18:36.949944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.173713 (+223769us) service_pool.cc:224] Handling call
0709 12:18:37.173969 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.608529 (+1434560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:38.608575 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:38.664329 (+ 55754us) log.cc:844] Serialized 26 byte log entry
0709 12:18:38.923386 (+259057us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:38.923430 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.923504 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.982645 (+ 59141us) raft_consensus.cc:1685] finished
0709 12:18:38.982663 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.982796 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35758,"raft.queue_time_us":99595,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":20987,"thread_start_us":1649637,"threads_started":2,"wal-append.queue_time_us":294557}
I20260709 12:18:38.990664 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.991573 25785 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:38.987516 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:38.992964 25633 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:38.993986 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 585) took 1833 ms. Trace:
I20260709 12:18:38.996624 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.997272 25780 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:38.983810 28427 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:38.762125 26338 rpcz_store.cc:276] 0709 12:18:36.891785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.892823 (+  1038us) service_pool.cc:224] Handling call
0709 12:18:36.972097 (+ 79274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.061508 (+ 89411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.077380 (+ 15872us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:37.165999 (+ 88619us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.831977 (+665978us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:37.843515 (+ 11538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.860151 (+ 16636us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.721271 (+861120us) raft_consensus.cc:1685] finished
0709 12:18:38.721304 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.721494 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":881,"raft.queue_time_us":13186,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6409,"thread_start_us":273720,"threads_started":1,"wal-append.queue_time_us":1474096}
I20260709 12:18:38.995065 25634 rpcz_store.cc:276] 0709 12:18:37.161043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.161176 (+   133us) service_pool.cc:224] Handling call
0709 12:18:37.189805 (+ 28629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.238697 (+ 48892us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:37.406742 (+168045us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.952643 (+1545901us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.960580 (+  7937us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:38.976487 (+ 15907us) log.cc:844] Serialized 26 byte log entry
0709 12:18:38.976677 (+   190us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:38.976711 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.976801 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.993795 (+ 16994us) raft_consensus.cc:1685] finished
0709 12:18:38.993814 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.993933 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83413,"prepare.queue_time_us":194650,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":69030,"raft.run_cpu_time_us":504,"raft.run_wall_time_us":42834,"thread_start_us":1614114,"threads_started":3,"wal-append.queue_time_us":1318415}
W20260709 12:18:39.003015 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 193112us
W20260709 12:18:39.007382 26336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 511) took 1800 ms (client timeout 2046 ms). Trace:
W20260709 12:18:39.007858 26336 rpcz_store.cc:269] 0709 12:18:37.206839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.344706 (+137867us) service_pool.cc:224] Handling call
0709 12:18:39.007189 (+1662483us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540061d0d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:39.007315 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200543616}
I20260709 12:18:38.975593 27936 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.922561 28139 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:38.974937 28089 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:39.014214 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.014837 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.015326 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:38.947801 28258 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:38.977588 28369 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:38.990829 28346 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.018754 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:38.982934 28551 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:39.015367 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 466613us
W20260709 12:18:39.046113 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2760956670001633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 12:18:39.048214 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:39.024469 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 89227.9us
I20260709 12:18:38.997781 28519 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:39.041841 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.036037 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.013785 28345 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.010704 28483 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:39.046852 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.778s (SENT) [suppressed 7 similar messages]
I20260709 12:18:39.057689 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.060212 28139 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:38.990262 28517 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:39.060353 28530 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.595s	user 0.000s	sys 0.016s
I20260709 12:18:39.044358 28089 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.035804 28422 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.079265 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2195) took 1650 ms. Trace:
I20260709 12:18:39.084357 25712 rpcz_store.cc:276] 0709 12:18:37.429009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.477813 (+ 48804us) service_pool.cc:224] Handling call
0709 12:18:37.501150 (+ 23337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.657187 (+156037us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:39.018084 (+1360897us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.078783 (+ 60699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:39.078858 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:39.078923 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.079010 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.079196 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":17502,"threads_started":1}
I20260709 12:18:39.083504 28345 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.089928 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.090312 28345 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.057293 28419 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.091347 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 508) took 1910 ms. Trace:
I20260709 12:18:39.054708 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:39.100664 25676 connection.cc:256] RPC call timeout handler was delayed by -1.186852786000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
I20260709 12:18:39.018965 27936 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:39.054396 28441 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:39.105662 26344 rpcz_store.cc:276] 0709 12:18:37.180882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.758609 (+1577727us) service_pool.cc:224] Handling call
0709 12:18:38.759182 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:38.762718 (+  3536us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:38.951185 (+188467us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.084538 (+133353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:39.084598 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:39.091059 (+  6461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.091142 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.091279 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":328385,"thread_start_us":309287,"threads_started":3}
W20260709 12:18:39.111052 25676 connection.cc:256] RPC call timeout handler was delayed by -1.178685668000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 12:18:39.047608 28369 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:39.117377 25676 connection.cc:256] RPC call timeout handler was delayed by -1.157152786000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 12:18:39.121673 25676 connection.cc:256] RPC call timeout handler was delayed by -1.1526527860000897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 12:18:39.120299 25751 connection.cc:256] RPC call timeout handler was delayed by -2.2066488210002717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2298s for a response
W20260709 12:18:39.122170 25676 connection.cc:256] RPC call timeout handler was delayed by -1.1120856679999633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 12:18:39.122359 25751 connection.cc:256] RPC call timeout handler was delayed by -1.1691929200001141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 12:18:39.122601 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0680527860001803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
W20260709 12:18:39.122792 25751 connection.cc:256] RPC call timeout handler was delayed by -1.1529929200000879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 12:18:39.123047 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0511039110001548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 12:18:39.123148 25751 connection.cc:256] RPC call timeout handler was delayed by -1.1034929200000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
I20260709 12:18:39.127449 27936 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:39.103538 28560 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.131302 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:39.054813 28346 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.138527 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.150619 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 582) took 2058 ms. Trace:
W20260709 12:18:39.153204 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 72913.1us
I20260709 12:18:39.154422 25785 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:39.154861 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1570) took 1761 ms (client timeout 2099 ms). Trace:
W20260709 12:18:39.155282 25779 rpcz_store.cc:269] 0709 12:18:37.393325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.393494 (+   169us) service_pool.cc:224] Handling call
0709 12:18:39.154688 (+1761194us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400099f54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:39.154798 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3131368832}
I20260709 12:18:39.157745 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.161273 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.161960 25632 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:39.151116 25635 rpcz_store.cc:276] 0709 12:18:37.092444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.092618 (+   174us) service_pool.cc:224] Handling call
0709 12:18:37.099023 (+  6405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:37.099624 (+   601us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:37.099666 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:37.100506 (+   840us) log.cc:844] Serialized 52 byte log entry
0709 12:18:37.300477 (+199971us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:37.300546 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.300649 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.059697 (+1759048us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:39.145834 (+ 86137us) raft_consensus.cc:1685] finished
0709 12:18:39.145859 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.150533 (+  4674us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14031,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":15471488,"thread_start_us":177287,"threads_started":1,"wal-append.queue_time_us":300647}
I20260709 12:18:39.166618 25779 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:18:39.167682 28563 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:18:39.168170 25779 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:39.169436 28562 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 12:18:39.170384 28566 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:18:39.113724 28089 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.179107 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:39.182121 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:39.157277 25778 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:39.187204 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:39.113291 28419 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.200439 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.200718 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.201252 25635 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:18:39.211306 28576 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 12:18:39.213724 28572 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:39.216032 28419 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.221240 25778 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:39.222712 28258 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:39.224829 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.236905 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1577) took 1216 ms. Trace:
I20260709 12:18:39.197644 28440 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:39.142355 28560 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.237411 25787 rpcz_store.cc:276] 0709 12:18:38.020751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.819312 (+798561us) service_pool.cc:224] Handling call
0709 12:18:38.902957 (+ 83645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.925418 (+ 22461us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:38.925452 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:38.949421 (+ 23969us) log.cc:844] Serialized 26 byte log entry
0709 12:18:39.091850 (+142429us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:39.091892 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.091960 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.236645 (+144685us) raft_consensus.cc:1685] finished
0709 12:18:39.236666 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.236835 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27380,"raft.queue_time_us":195,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":6898,"thread_start_us":129159,"threads_started":1,"wal-append.queue_time_us":208221}
I20260709 12:18:39.241268 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 509) took 2040 ms. Trace:
I20260709 12:18:39.242910 26341 rpcz_store.cc:276] 0709 12:18:37.200815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.785835 (+1585020us) service_pool.cc:224] Handling call
0709 12:18:38.789019 (+  3184us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:38.798235 (+  9216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:38.798268 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:38.818363 (+ 20095us) log.cc:844] Serialized 590 byte log entry
0709 12:18:38.926289 (+107926us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:38.926325 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.926392 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.241044 (+314652us) raft_consensus.cc:1685] finished
0709 12:18:39.241071 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.241210 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5501,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":5357,"spinlock_wait_cycles":41344,"thread_start_us":104141,"threads_started":1,"wal-append.queue_time_us":319154}
W20260709 12:18:39.251260 25677 connection.cc:256] RPC call timeout handler was delayed by -1.1587777550003011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 12:18:39.252317 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.246572 28560 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.241227 28440 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.259219 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.264595 25635 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:39.259969 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:39.268679 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 216602us
I20260709 12:18:39.271525 25787 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:18:39.276073 25637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 581) took 2184 ms (client timeout 1984 ms). Trace:
W20260709 12:18:39.276540 25637 rpcz_store.cc:269] 0709 12:18:37.092143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.092247 (+   104us) service_pool.cc:224] Handling call
0709 12:18:39.275893 (+2183646us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400364554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:39.275997 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324577152}
W20260709 12:18:39.285912 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54328.7us
I20260709 12:18:39.241386 28577 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:39.290014 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.289932 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.315032 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.317754 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.271162 28440 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:39.321018 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.322142 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:39.323388 28550 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.561s	user 0.003s	sys 0.000s
I20260709 12:18:39.325613 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.330951 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.331544 25787 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:39.326887 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.328477 25786 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:39.333297 25632 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:39.334642 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.273554 28345 raft_consensus.cc:2955] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:39.334756 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.334508 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1573) took 1831 ms. Trace:
I20260709 12:18:39.315188 28560 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:39.340796 25636 raft_consensus.cc:2368] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:39.326016 25705 raft_consensus.cc:2955] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:39.337301 25786 rpcz_store.cc:276] 0709 12:18:37.503348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.503509 (+   161us) service_pool.cc:224] Handling call
0709 12:18:39.320867 (+1817358us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:39.334416 (+ 13549us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40776960}
I20260709 12:18:39.339000 28408 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:39.339771 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.344202 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.345440 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.340459 28560 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.346091 25638 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:39.346735 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.347852 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.348541 25713 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:39.349239 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.316169 28577 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:39.351660 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.350217 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:39.354247 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 174058us
I20260709 12:18:39.315821 28583 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:39.371886 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.352931 25630 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:39.374742 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.380767 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.381669 25631 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:39.385917 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.386693 25711 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:39.388069 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.353026 28560 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:39.391271 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:39.373885 28555 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.585s	user 0.000s	sys 0.002s
I20260709 12:18:39.394454 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.387001 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:39.345093 28408 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:39.402529 28584 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:18:39.405424 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.406371 25634 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:39.407361 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.408193 25634 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:39.409027 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.417200 28408 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:39.418561 28587 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:39.419057 28587 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.419378 28587 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:39.410198 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:39.423555 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 163643us
I20260709 12:18:39.306173 28551 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.428544 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.430455 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.431458 25677 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:39.432078 28594 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:39.368086 28583 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:39.433300 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.387351 28089 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.441139 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:39.446333 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 175982us
I20260709 12:18:39.472159 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.510731 28440 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:39.590862 25711 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:39.543329 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:39.650229 25634 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:39.650781 25711 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:39.651305 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.486613 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:39.651762 28440 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:39.652201 25634 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:39.437893 28583 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:39.652561 25631 raft_consensus.cc:2410] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:39.446929 28577 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:39.653512 25634 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:39.540990 28593 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:39.552060 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 188020us
W20260709 12:18:39.594853 25601 connection.cc:256] RPC call timeout handler was delayed by -2.236472434999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260709 12:18:39.491626 25712 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:39.655782 25601 connection.cc:256] RPC call timeout handler was delayed by -2.1887724349999189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:18:39.657428 25601 connection.cc:256] RPC call timeout handler was delayed by -1.9421724349999749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 12:18:39.657881 25601 connection.cc:256] RPC call timeout handler was delayed by -1.8053724349999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
I20260709 12:18:39.596927 28517 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:39.615288 28369 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.640463 25638 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:39.609648 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1574) took 1931 ms. Trace:
I20260709 12:18:39.649371 25636 raft_consensus.cc:2410] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:18:39.601441 28590 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:18:39.655766 25631 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:39.648130 28338 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:39.658222 25712 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: [4.3-6.4]
W20260709 12:18:39.602125 26335 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.003s
W20260709 12:18:39.641338 28139 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.003s
I20260709 12:18:39.610947 28588 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:39.667320 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 235970us
W20260709 12:18:39.667441 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 195862us
I20260709 12:18:39.668231 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:39.672416 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:39.600623 28595 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:39.674646 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.659198 28585 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.660452 25784 rpcz_store.cc:276] 0709 12:18:37.678344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.678471 (+   127us) service_pool.cc:224] Handling call
0709 12:18:37.850395 (+171924us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:37.892025 (+ 41630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.892055 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:37.924092 (+ 32037us) log.cc:844] Serialized 52 byte log entry
0709 12:18:38.617595 (+693503us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:38.927478 (+309883us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.945809 (+ 18331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.945922 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.338925 (+393003us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:39.603780 (+264855us) raft_consensus.cc:1685] finished
0709 12:18:39.609374 (+  5594us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.609573 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35231,"prepare.queue_time_us":360061,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":45654,"raft.run_cpu_time_us":1028,"raft.run_wall_time_us":16802,"spinlock_wait_cycles":38289408,"thread_start_us":879033,"threads_started":2,"wal-append.queue_time_us":756252}
W20260709 12:18:39.685120 26307 connection.cc:256] RPC call timeout handler was delayed by -2.1907385780000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:18:39.686372 25634 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:39.685624 26307 connection.cc:256] RPC call timeout handler was delayed by -2.1907385780000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:18:39.687615 25636 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:39.677999 28519 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:39.432788 28594 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:39.720108 26307 connection.cc:256] RPC call timeout handler was delayed by -2.0386385780000182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 12:18:39.720830 26307 connection.cc:256] RPC call timeout handler was delayed by -1.8856385779999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 12:18:39.721261 26307 connection.cc:256] RPC call timeout handler was delayed by -1.8847385779999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260709 12:18:39.720880 25783 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:18:39.721673 26307 connection.cc:256] RPC call timeout handler was delayed by -1.8640385779999633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
I20260709 12:18:39.722940 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:39.722196 26307 connection.cc:256] RPC call timeout handler was delayed by -1.4752213320000465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
I20260709 12:18:39.668447 28517 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.655064 28593 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.729801 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1571) took 2304 ms. Trace:
I20260709 12:18:39.730011 25636 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:18:39.734356 25634 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:39.733982 25636 pending_rounds.cc:85] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 2
I20260709 12:18:39.746229 25636 pending_rounds.cc:107] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:18:39.746835 25636 raft_consensus.cc:2889] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:39.728703 28595 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:39.733124 25782 rpcz_store.cc:276] 0709 12:18:37.425127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.425314 (+   187us) service_pool.cc:224] Handling call
0709 12:18:37.518951 (+ 93637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.685207 (+166256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.729598 (+2044391us) spinlock_profiling.cc:243] Waited 79 ms on lock 0x7b54000b04d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:39.729738 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181646592}
W20260709 12:18:39.751392 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 95563us
I20260709 12:18:39.754334 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:39.755517 28605 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:39.756171 28605 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:39.757339 28597 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:18:39.763296 28580 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:39.764729 28560 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.765281 28593 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.765908 28593 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:39.767890 25707 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:18:39.769529 25712 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:39.688649 28585 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.733103 28577 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:18:39.794436 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 507) took 2613 ms. Trace:
W20260709 12:18:39.798760 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 109843us
I20260709 12:18:39.799027 26335 rpcz_store.cc:276] 0709 12:18:37.180666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.498133 (+317467us) service_pool.cc:224] Handling call
0709 12:18:37.519182 (+ 21049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.519524 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:39.676763 (+2157239us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.787032 (+110269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:39.791097 (+  4065us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:39.794135 (+  3038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.794211 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.794330 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8516,"thread_start_us":47265,"threads_started":1}
I20260709 12:18:39.794669 28609 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:18:39.799770 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:39.796836 28583 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.804788 28583 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.798197 28560 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.796034 28398 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:18:39.676241 28588 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:39.787024 28592 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.189s	user 0.000s	sys 0.002s
I20260709 12:18:39.820111 28588 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:39.820674 28588 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.821081 28588 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.772897 28595 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:39.785161 28604 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:39.773665 28607 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:39.763064 28605 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.851191 25677 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:39.827590 28560 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:39.868990 28517 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:39.896054 28588 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:39.897518 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2199) took 2050 ms (client timeout 1838 ms). Trace:
I20260709 12:18:39.897800 25710 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:18:39.898036 25706 rpcz_store.cc:269] 0709 12:18:37.846891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.855331 (+  8440us) service_pool.cc:224] Handling call
0709 12:18:39.897209 (+2041878us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400203d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:39.897399 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508380928}
I20260709 12:18:39.854592 28560 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:39.906672 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 150386us
I20260709 12:18:39.831988 28604 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.910024 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.914034 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:39.914947 25712 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:39.916108 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.916770 25710 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:18:39.923482 25750 connection.cc:256] RPC call timeout handler was delayed by -1.2344903290002094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
I20260709 12:18:39.908941 28560 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:39.937093 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.937942 25787 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:18:39.928390 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.831712 28607 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:39.929996 25706 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:18:39.945205 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 94116us
I20260709 12:18:39.937960 25784 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:39.939158 25782 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:39.913028 28371 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.947340 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.969439 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:39.970873 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 181965us
I20260709 12:18:39.936070 25712 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:39.986858 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:39.991356 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.988529 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:39.995407 25781 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:39.994072 25787 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
W20260709 12:18:39.992581 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 256710us
I20260709 12:18:39.997824 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.945586 28594 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:40.000281 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:39.976097 28408 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.003s
I20260709 12:18:40.008841 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:40.009531 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 622564us
I20260709 12:18:40.040755 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:39.977660 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:40.046633 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 140274us
I20260709 12:18:39.991247 28577 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:40.085418 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.086292 26339 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 5. Ops: []
I20260709 12:18:40.050828 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.052040 28585 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:40.053050 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.044749 27534 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:40.056763 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.059237 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:40.086099 26341 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:40.091660 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:40.059746 28604 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:40.061601 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.102898 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.104205 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:40.098197 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3990459530000408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
I20260709 12:18:40.111503 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.112425 26339 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:40.116379 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:40.118891 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.130059 26343 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:40.133061 25782 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:18:39.995048 28371 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.137406 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.137410 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:39.941151 28595 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.086459 28577 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:40.087671 28408 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.090417 27534 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:40.163409 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.164551 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.193596 25787 raft_consensus.cc:2410] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:40.090692 28559 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:40.111196 28290 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:40.141675 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:40.140265 28427 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.102s	user 0.010s	sys 0.005s
W20260709 12:18:40.151083 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 100396us
I20260709 12:18:40.214033 26307 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.141075 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.155881 28457 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:40.157509 28604 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:40.131209 28264 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:18:40.171942 28525 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 2.715s	user 0.020s	sys 0.006s
I20260709 12:18:40.131726 28558 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:40.157747 28585 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:40.137540 28139 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:18:40.198932 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.199260 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.199606 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1589) took 1428 ms. Trace:
I20260709 12:18:40.099905 28089 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:40.205397 28520 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:40.237174 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.205502 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:40.208907 28415 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:40.164422 28612 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 12:18:40.190769 28614 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.055s	user 0.001s	sys 0.001s
I20260709 12:18:40.231873 28290 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:40.158936 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.131668 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:40.215983 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.263490 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.194695 27534 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:40.135541 28459 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:40.216876 25711 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:40.217381 28578 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:40.182083 28420 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:40.218397 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 77378.2us
I20260709 12:18:40.267019 26339 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:40.267570 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.195143 28577 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:40.219738 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 295611us
I20260709 12:18:40.219048 28427 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.235224 25783 rpcz_store.cc:276] 0709 12:18:38.771103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.791253 (+ 20150us) service_pool.cc:224] Handling call
0709 12:18:40.199393 (+1408140us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b540011f154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:40.199532 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2017181440}
I20260709 12:18:40.238102 25784 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:40.239814 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:40.236850 28585 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.237574 28264 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:40.248629 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:40.250599 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 145493us
W20260709 12:18:40.275261 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3500029819999781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
I20260709 12:18:40.275406 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.251346 25787 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:40.246860 25706 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:40.137571 27936 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:18:40.255224 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.264248 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.237704 28520 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:40.260524 28483 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:40.265983 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 258788us
I20260709 12:18:40.268437 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.281764 26344 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:18:40.270007 25750 connection.cc:256] RPC call timeout handler was delayed by -1.0580903290001515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
I20260709 12:18:40.270328 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:40.282143 25750 connection.cc:256] RPC call timeout handler was delayed by -1.0211903290000919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260709 12:18:40.272045 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2192) took 2843 ms. Trace:
I20260709 12:18:40.278088 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.273794 28415 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:40.283072 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:40.283078 28457 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:40.286759 25676 connection.cc:256] RPC call timeout handler was delayed by -1.3252459529999214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 12:18:40.287662 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:40.287810 25638 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:40.288956 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.289265 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:40.288309 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.271451 28558 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.291359 25677 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:40.293551 25784 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:40.293964 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:40.295697 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2343379139997523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 12:18:40.296111 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2217459529999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:18:40.296427 25676 connection.cc:256] RPC call timeout handler was delayed by -1.2217459529999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260709 12:18:40.298828 28593 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:40.299343 28593 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:40.299788 28593 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:40.299979 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 1964) took 1130 ms. Trace:
I20260709 12:18:40.300737 28459 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:40.301718 28570 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:40.301174 28520 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.302161 28570 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.294777 26335 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.305334 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1587) took 1574 ms. Trace:
W20260709 12:18:40.305904 28611 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.470s	user 0.001s	sys 0.001s
W20260709 12:18:40.307253 28635 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:18:40.305815 25779 rpcz_store.cc:276] 0709 12:18:38.731124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.168852 (+437728us) service_pool.cc:224] Handling call
0709 12:18:39.169167 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.976780 (+807613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.976811 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:40.003367 (+ 26556us) log.cc:844] Serialized 26 byte log entry
0709 12:18:40.256041 (+252674us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:40.256108 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.256212 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.305040 (+ 48828us) raft_consensus.cc:1685] finished
0709 12:18:40.305067 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.305244 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":851272,"raft.queue_time_us":807564,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":9430,"thread_start_us":198443,"threads_started":1,"wal-append.queue_time_us":250801}
I20260709 12:18:40.309657 28607 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:40.315126 28264 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:40.322366 28621 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:18:40.322937 28457 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:40.280753 26338 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:40.298544 28604 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:18:40.336432 25676 connection.cc:256] RPC call timeout handler was delayed by -1.1947459529999378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
W20260709 12:18:40.343210 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 60201.2us
I20260709 12:18:40.283856 28369 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:40.344179 25676 connection.cc:256] RPC call timeout handler was delayed by -1.1640457399998922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2258s for a response
W20260709 12:18:40.344630 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0912697480000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.38570000000000004s for a response
I20260709 12:18:40.300503 25631 rpcz_store.cc:276] 0709 12:18:39.169183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.678162 (+508979us) service_pool.cc:224] Handling call
0709 12:18:39.694178 (+ 16016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.698809 (+  4631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.698839 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:39.731677 (+ 32838us) log.cc:844] Serialized 26 byte log entry
0709 12:18:39.859861 (+128184us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:39.859903 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.859987 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.299753 (+439766us) raft_consensus.cc:1685] finished
0709 12:18:40.299774 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.299917 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15839,"raft.queue_time_us":4214,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":26251,"thread_start_us":114954,"threads_started":1,"wal-append.queue_time_us":245859}
W20260709 12:18:40.344933 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0120459529998698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
I20260709 12:18:40.344844 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:40.284552 28290 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:40.348410 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:40.350056 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 56174us
I20260709 12:18:40.241791 25778 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:40.284294 25708 rpcz_store.cc:276] 0709 12:18:37.428167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.440750 (+ 12583us) service_pool.cc:224] Handling call
0709 12:18:37.470223 (+ 29473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.814716 (+344493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.814753 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:37.855598 (+ 40845us) log.cc:844] Serialized 26 byte log entry
0709 12:18:40.038065 (+2182467us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b18004151e0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:18:40.252453 (+214388us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:40.252496 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.252565 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.263836 (+ 11271us) raft_consensus.cc:1685] finished
0709 12:18:40.263855 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.271979 (+  8124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84079,"raft.queue_time_us":242047,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":16569,"spinlock_wait_cycles":4276608,"thread_start_us":214191,"threads_started":1,"wal-append.queue_time_us":2351277}
I20260709 12:18:40.271360 28420 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.355095 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:40.303078 28628 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:40.358572 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:40.359293 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71062.2us
I20260709 12:18:40.361372 26337 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:40.364945 28459 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:40.365583 28641 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:40.293152 28578 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:40.302577 28520 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:40.336855 28570 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.370363 25782 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:40.377261 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.385449 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.395033 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 588) took 1569 ms. Trace:
I20260709 12:18:40.395154 26340 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:40.394368 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.397324 25779 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:40.397898 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.285430 28483 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.396621 26342 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:40.395587 25638 rpcz_store.cc:276] 0709 12:18:38.825302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.660521 (+835219us) service_pool.cc:224] Handling call
0709 12:18:39.660820 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.288576 (+627756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.394709 (+106133us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b54001d65d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:40.394907 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441577344}
I20260709 12:18:40.408876 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.409852 25634 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:40.360000 27936 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:40.328157 28258 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 12:18:40.375083 28338 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.706s	user 0.007s	sys 0.007s
W20260709 12:18:40.392832 28619 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:18:40.373050 28607 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:40.420063 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:40.421945 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 66627.7us
I20260709 12:18:40.423192 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.424703 25631 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:40.436298 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:40.436491 25779 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:40.437469 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:40.348542 28615 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:18:40.335695 28558 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.449759 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.369166 28610 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
W20260709 12:18:40.449838 25780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1599) took 1449 ms (client timeout 1752 ms). Trace:
W20260709 12:18:40.465118 25780 rpcz_store.cc:269] 0709 12:18:39.004792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.018819 (+ 14027us) service_pool.cc:224] Handling call
0709 12:18:40.419824 (+1401005us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:18:40.449776 (+ 29952us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10183552}
I20260709 12:18:40.419468 28264 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:40.433373 27923 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.402556 28415 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:18:40.476336 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 1955) took 1310 ms. Trace:
I20260709 12:18:40.416169 28398 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:18:40.412091 28290 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:18:40.476822 25630 rpcz_store.cc:276] 0709 12:18:39.166387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.224423 (+ 58036us) service_pool.cc:224] Handling call
0709 12:18:40.476139 (+1251716us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54002805d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:40.476265 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293082880}
I20260709 12:18:40.434315 28559 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:40.411461 28636 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 12:18:40.482244 27923 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.523073 28427 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:40.488210 28459 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.489053 27534 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.529362 26342 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:40.543067 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.545689 27534 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:40.470106 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 86037.4us
I20260709 12:18:40.471681 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.485507 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.548820 26337 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:40.485383 28559 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:40.489988 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2135) took 1169 ms. Trace:
I20260709 12:18:40.490381 28420 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:40.555994 25635 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:40.434056 28440 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:40.495054 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:40.569126 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.495710 28625 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:40.502745 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.503762 28551 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:40.506659 28603 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:40.508693 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.582340 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.510164 28490 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:18:40.514218 28560 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 12:18:40.514938 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.574739 28610 consensus_queue.cc:1243] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:18:40.602975 25631 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:40.602797 28560 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.605510 28560 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:40.517874 28558 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:40.478039 28408 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:40.475477 28587 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.056s	user 0.011s	sys 0.004s
I20260709 12:18:40.473440 28607 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:40.477265 28346 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:40.609438 28587 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.609508 28559 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.611081 28559 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:40.611385 28593 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.612375 25705 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:40.552134 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 190074us
I20260709 12:18:40.495076 25782 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:40.570571 25632 raft_consensus.cc:2410] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:40.613878 28593 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:40.549602 25713 rpcz_store.cc:276] 0709 12:18:39.320507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.433062 (+112555us) service_pool.cc:224] Handling call
0709 12:18:39.469456 (+ 36394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:39.574929 (+105473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.574983 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:39.667030 (+ 92047us) log.cc:844] Serialized 52 byte log entry
0709 12:18:39.742706 (+ 75676us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:39.760334 (+ 17628us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.930673 (+170339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.930747 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.305299 (+374552us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:40.489644 (+184345us) raft_consensus.cc:1685] finished
0709 12:18:40.489742 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.489921 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19909,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":184555,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149,"spinlock_wait_cycles":35147520,"thread_start_us":204542,"threads_started":2,"wal-append.queue_time_us":115226}
I20260709 12:18:40.521754 28483 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:40.619709 26335 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:18:40.573997 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 205113us
I20260709 12:18:40.596657 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.635526 25631 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:40.611788 28587 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:40.623067 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2139) took 1301 ms. Trace:
I20260709 12:18:40.586426 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:40.577390 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49710.7us
I20260709 12:18:40.597208 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.577864 25780 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:40.643700 25711 rpcz_store.cc:276] 0709 12:18:39.321508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.655418 (+333910us) service_pool.cc:224] Handling call
0709 12:18:39.659685 (+  4267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.264336 (+604651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.622797 (+358461us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400235d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:40.623000 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280822016}
I20260709 12:18:40.577224 25785 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:40.644430 25634 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:18:40.651258 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 69471.8us
I20260709 12:18:40.695502 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.704545 26337 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:40.581749 28603 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:40.709872 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 272451us
I20260709 12:18:40.593428 28490 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:40.708601 25636 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:18:40.718950 25634 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:40.712016 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:40.591071 28598 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:40.709765 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 96647.7us
I20260709 12:18:40.721417 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.724897 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.728480 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.729295 25779 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:40.732334 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.570930 28647 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:40.743513 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:40.745460 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.735337 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.735973 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.738623 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.741099 27923 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:40.747925 25779 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:40.762446 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.696141 28585 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:40.766453 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.770141 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.770834 26339 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:40.772311 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:40.772989 25779 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:40.789804 25782 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:18:40.790387 25782 pending_rounds.cc:85] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 5
I20260709 12:18:40.790673 25782 pending_rounds.cc:107] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:18:40.790947 25782 raft_consensus.cc:2889] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.734576 28490 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:40.810042 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.810123 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2213) took 1539 ms. Trace:
I20260709 12:18:40.795153 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.811276 25637 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:40.796399 28600 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.068s	user 0.001s	sys 0.003s
I20260709 12:18:40.796510 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.766084 28598 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.798081 25787 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:40.575016 28654 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.203s
I20260709 12:18:40.734007 28649 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:40.799109 25783 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:40.799688 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.801074 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:40.801649 28408 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:40.620244 28625 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.763633 28587 raft_consensus.cc:2764] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:18:40.717960 28369 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 12:18:40.703299 28483 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 12:18:40.810884 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.810585 25707 rpcz_store.cc:276] 0709 12:18:39.271024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.271268 (+   244us) service_pool.cc:224] Handling call
0709 12:18:39.315077 (+ 43809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.770324 (+455247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.809853 (+1039529us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b5400104d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:40.810057 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038293888}
I20260709 12:18:40.812770 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.812878 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.696784 28577 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.815830 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:40.857209 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:40.816236 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.818835 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.833882 28623 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.858829 25781 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:40.859022 28623 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.859467 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.860061 25781 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:40.860354 28369 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:40.860698 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.860862 28369 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:40.835628 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.861279 25781 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:40.861515 28623 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:40.861763 25787 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:40.808156 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.863615 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.864285 26343 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:40.865334 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:40.865657 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.866600 28593 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:40.867441 28616 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:40.867890 28616 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:40.869029 28616 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:40.870183 28638 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.420s	user 0.000s	sys 0.002s
I20260709 12:18:40.864377 25710 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:40.871343 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.872876 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.875660 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.875754 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:40.877704 28598 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:40.878759 28371 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:40.879426 28371 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:40.879835 28371 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.880236 28371 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:40.881554 28616 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:40.838678 28603 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.891600 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:40.892536 28603 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:40.893507 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:40.746122 28570 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:40.894958 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:40.895759 25704 raft_consensus.cc:2368] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:40.896765 28483 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:40.897178 28483 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:40.898010 28483 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:40.899005 28644 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:18:40.899418 28483 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:40.901206 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:40.901831 25707 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:40.896809 25706 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:40.902601 26340 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:40.898031 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:40.904140 25708 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:40.849920 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:40.892482 25711 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:40.905164 25783 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
W20260709 12:18:40.905539 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 68834.8us
I20260709 12:18:40.837990 28578 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:18:40.851943 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.778301 28650 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:18:40.779484 28617 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 12:18:40.813637 28481 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:40.859369 28574 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:40.862243 28415 raft_consensus.cc:2764] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:18:40.836470 25786 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:40.866878 28577 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:40.851729 25782 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:18:40.907317 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:40.874015 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.968873 25633 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:40.970151 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.909938 25752 leader_election.cc:400] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:40.970909 25633 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.971246 25752 leader_election.cc:403] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:18:40.971942 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62129.8us
I20260709 12:18:40.972602 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:40.973667 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:40.866631 25630 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:18:40.916620 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48893us
I20260709 12:18:40.984230 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:40.985176 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 80836.3us
W20260709 12:18:40.986569 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49722.7us
W20260709 12:18:40.937323 26338 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.005s
I20260709 12:18:40.939262 25637 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:40.940026 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:40.943159 25711 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:40.944150 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:40.945909 25779 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 3.
I20260709 12:18:40.961521 26339 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:40.961747 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:40.896795 28570 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:40.964699 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:40.913792 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.992002 26338 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:40.999580 25632 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:18:40.965773 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:40.993826 25706 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:41.000216 25709 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:40.996202 25712 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:40.964327 28623 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:18:40.946403 28651 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.936375 28647 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:18:41.002784 28657 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:41.002825 28420 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:18:41.002934 27936 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:41.005285 27936 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.005668 27936 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.902663 28649 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.008486 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.009862 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.010774 25638 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:41.011621 27923 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.012285 27923 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.012755 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:40.996150 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.861933 28369 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.018698 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.020057 25704 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:41.022001 25629 raft_consensus.cc:2368] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:41.022680 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:41.026010 25632 raft_consensus.cc:2368] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:40.962458 28558 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:41.038576 28415 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:41.039518 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:40.944450 28258 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:18:41.043128 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.056859 25632 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:41.052348 26340 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:40.835474 28607 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:40.997649 28264 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:41.014775 28658 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.967180 28598 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:18:41.072484 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.090281 25629 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:41.074611 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:40.991868 28594 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:18:41.078388 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:41.082856 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 69122.1us
I20260709 12:18:41.063234 28369 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:41.063476 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:41.059648 28415 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.027704 28574 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:18:41.098237 25630 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:41.076872 28264 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:41.097499 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 110271us
I20260709 12:18:41.058001 28558 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.066468 28604 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:41.062879 28610 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:18:41.103207 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.103870 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.110390 25637 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:41.082298 25633 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:41.111449 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:41.056349 28651 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.133380 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:41.102772 28415 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:41.159112 25633 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:41.117681 26336 raft_consensus.cc:2410] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:41.115566 25629 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:41.105502 28264 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.161093 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.098055 28398 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:18:41.105952 28615 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:41.200044 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.108670 28583 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:41.203078 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 91755.7us
I20260709 12:18:41.111436 28636 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:18:41.106244 28625 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:41.122776 28596 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:41.212529 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:41.112633 27923 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:41.213745 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 269636us
I20260709 12:18:41.101675 28660 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 12:18:41.214586 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2218) took 1540 ms. Trace:
I20260709 12:18:41.210064 28615 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:41.229657 25705 rpcz_store.cc:276] 0709 12:18:39.674168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.674325 (+   157us) service_pool.cc:224] Handling call
0709 12:18:41.214298 (+1539973us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54003ac854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:41.214506 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2146527488}
I20260709 12:18:41.207998 28650 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:41.225386 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:41.211021 28583 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:41.211316 25637 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:18:41.238142 28669 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:18:41.238617 28558 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.233489 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:41.252651 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:41.257668 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.229703 28577 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:41.232815 28660 consensus_queue.cc:1243] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:41.275063 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:41.252084 28650 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:41.279481 28652 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:41.279821 27936 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.280527 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:41.240376 28615 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.281659 28615 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:41.282114 27936 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.282665 28574 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:41.283252 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.287189 28595 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:41.274693 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48907.4us
I20260709 12:18:41.260607 28558 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.293720 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:41.297652 28626 raft_consensus.cc:2764] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:41.299137 28371 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.300370 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.300802 28651 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:41.301187 25782 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:41.302716 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.291105 28660 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:41.308449 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:41.308094 28490 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.309022 26344 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:41.292677 28595 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.309613 28595 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.287288 28681 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:41.321491 25782 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:18:41.322022 25706 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:41.286896 28596 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:41.338275 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.338837 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2081) took 1162 ms. Trace:
I20260709 12:18:41.345906 28490 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:41.349695 28673 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:41.354741 28649 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.355502 28371 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.319303 28558 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.362252 28585 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:41.362689 28585 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:41.363477 28585 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:41.364184 28559 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.003s
I20260709 12:18:41.364521 28585 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:41.310935 28625 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:18:41.333843 27936 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:41.365878 27936 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:41.366972 27936 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.373080 25782 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:41.378033 25780 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:41.379398 26344 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
W20260709 12:18:41.389134 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 65084.5us
I20260709 12:18:41.374217 28658 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:41.394779 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:41.396394 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:41.390393 28558 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.415874 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:41.417421 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:41.418140 26344 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:41.419231 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:41.407871 28415 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.374882 28583 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.396569 28636 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:18:41.341428 25778 rpcz_store.cc:276] 0709 12:18:40.176345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.177266 (+   921us) service_pool.cc:224] Handling call
0709 12:18:40.204028 (+ 26762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.223180 (+ 19152us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:40.901873 (+678693us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.290524 (+388651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:41.292484 (+  1960us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:41.295755 (+  3271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.321263 (+ 25508us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.335515 (+ 14252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129222,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":291778,"threads_started":2,"wal-append.queue_time_us":310606}
I20260709 12:18:41.381564 28625 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:18:41.404816 28596 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.419453 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:41.407955 28089 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:41.432221 28089 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:41.421846 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.433310 28089 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.434823 28089 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:41.436841 27936 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:41.437515 27936 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.438015 27936 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:41.438153 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 672) took 1435 ms. Trace:
I20260709 12:18:41.438925 28264 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.441125 28264 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:41.441372 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.442910 28623 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:41.443315 28623 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:41.438736 26335 rpcz_store.cc:276] 0709 12:18:40.002470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.048518 (+ 46048us) service_pool.cc:224] Handling call
0709 12:18:41.437940 (+1389422us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b540044c0d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:41.438088 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55857920}
I20260709 12:18:41.444180 28623 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.445521 28623 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:41.447815 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.449846 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.450731 26344 raft_consensus.cc:2393] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:41.451757 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:41.449960 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:41.454061 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 57754.9us
I20260709 12:18:41.455349 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:41.455276 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:41.456784 28658 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:41.458128 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 518142us
I20260709 12:18:41.459079 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:41.458842 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.460058 25705 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:41.460716 26308 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:41.461521 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:41.462193 25711 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.463040 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:41.465435 28649 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:41.466439 27936 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.467664 28625 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:41.470920 26340 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: [3.6-5.7]
W20260709 12:18:41.472314 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.473277 26337 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:41.474762 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.475764 27936 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.475895 26339 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:18:41.482162 28520 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:41.482656 28520 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:41.483009 28520 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:41.486654 28651 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:41.486786 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:41.487493 26342 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.489655 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.490850 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:41.491110 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:41.491487 26339 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:41.492300 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.494582 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.495079 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:41.496261 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:41.497553 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:41.470570 28264 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:18:41.493371 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.447105 28657 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:41.453022 28490 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:41.515789 28415 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:41.521600 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.523310 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.524403 26339 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:41.497939 28675 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:41.502568 28615 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:41.529331 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:41.526932 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.532294 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:41.532335 25705 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:41.538633 25711 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:18:41.541262 26342 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:18:41.509819 28650 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:41.517570 28583 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:41.516566 28580 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:41.518546 28620 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:41.524643 28368 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:41.563071 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.553823 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:41.519618 28490 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:41.573902 25705 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:41.529579 28560 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:41.582367 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.536458 28616 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:41.530689 28651 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:41.589205 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.591995 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.508850 28605 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:18:41.593672 25710 raft_consensus.cc:2410] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:41.596915 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:41.597271 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 43543us
I20260709 12:18:41.509546 28519 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:18:41.596452 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.542164 28481 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:41.566596 28517 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:41.610540 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.611515 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.612761 25633 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:41.566320 28658 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:41.620621 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.567695 28620 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:41.595888 28490 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:41.597848 28610 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:41.669967 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.635272 25709 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:41.670920 25707 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:41.671515 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:41.586052 28678 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:41.574573 28419 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:18:41.637305 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.593139 28675 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:41.593420 28415 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:41.614440 28580 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:41.612874 28558 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:41.693034 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.693082 25636 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:41.623225 28560 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:41.704453 28560 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.672258 28616 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:41.707768 28616 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.653069 28620 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:41.711068 28689 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:18:41.651366 28609 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:18:41.679956 28658 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.717265 28658 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.719596 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:41.643987 28625 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:41.702708 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.723685 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.724753 25710 raft_consensus.cc:2368] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:18:41.729173 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.722807 28440 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:41.733886 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.737224 25707 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:41.739763 28675 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.697376 28415 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.744613 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 601) took 1041 ms. Trace:
I20260709 12:18:41.744854 28415 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:41.745168 25638 rpcz_store.cc:276] 0709 12:18:40.703755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.707030 (+  3275us) service_pool.cc:224] Handling call
0709 12:18:41.744419 (+1037389us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400301bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:41.744549 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566466816}
I20260709 12:18:41.750233 25631 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:41.724439 28558 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:41.723637 28610 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:41.766477 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 203509us
W20260709 12:18:41.771790 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:41.781538 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 100903us
I20260709 12:18:41.785033 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:41.787205 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:41.791806 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:41.795428 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:41.782966 28692 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:18:41.806397 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:41.809082 26309 leader_election.cc:400] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:18:41.809723 25713 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:41.811371 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:41.812134 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:41.813028 26344 raft_consensus.cc:2368] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:41.802726 28598 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:41.816876 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:41.814893 28693 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.173s	user 0.003s	sys 0.001s
I20260709 12:18:41.807343 28415 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:41.840559 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:41.842015 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:41.872208 26340 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:41.873751 28699 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:41.875819 28520 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.877028 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:41.881386 28580 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:41.881847 28580 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.881963 28368 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:41.901482 28368 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:41.902403 28368 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.904022 28368 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:41.847137 26339 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:18:41.847772 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:41.870239 28585 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.003s
W20260709 12:18:41.850504 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.848289 28481 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:41.850967 28649 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.907220 28481 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:41.907812 28481 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:41.907531 28649 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.909934 28649 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:41.865420 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 742) took 1217 ms. Trace:
I20260709 12:18:41.911115 25633 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:41.911237 26336 rpcz_store.cc:276] 0709 12:18:40.647598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.860493 (+212895us) service_pool.cc:224] Handling call
0709 12:18:41.865195 (+1004702us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:41.865358 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588718464}
I20260709 12:18:41.816550 28598 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:41.912727 28598 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:41.864168 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1603) took 1158 ms. Trace:
I20260709 12:18:41.866798 27936 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:41.869772 28570 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:41.870599 28692 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:18:41.810135 28558 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.882292 28580 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:41.882863 28603 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:41.958768 28603 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:41.884236 28620 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:41.827999 28595 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
I20260709 12:18:41.844907 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:41.887359 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:41.891961 28519 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:41.888393 28419 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:18:41.810355 28594 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:41.815636 28610 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.970592 28419 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:41.971261 28419 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:41.971704 28620 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:41.972472 28419 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.956887 28490 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.895773 28415 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:41.898269 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:41.975610 25638 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:41.906391 26342 raft_consensus.cc:2368] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9.
I20260709 12:18:41.913868 25780 rpcz_store.cc:276] 0709 12:18:40.706664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.708241 (+  1577us) service_pool.cc:224] Handling call
0709 12:18:41.863954 (+1155713us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54002b5554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:41.864090 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427320064}
I20260709 12:18:41.985834 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.868691 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 527) took 1506 ms. Trace:
I20260709 12:18:41.941411 25708 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:18:41.987186 25708 pending_rounds.cc:85] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 3
I20260709 12:18:41.987552 25708 pending_rounds.cc:107] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:18:41.987864 25708 raft_consensus.cc:2889] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:41.943116 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:41.944963 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.949733 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:41.987007 26337 rpcz_store.cc:276] 0709 12:18:40.362582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.876720 (+514138us) service_pool.cc:224] Handling call
0709 12:18:41.473112 (+596392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.473622 (+   510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.868496 (+394874us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b540056ead0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:41.868628 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370960256}
I20260709 12:18:41.946703 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.939118 28139 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.953017 27534 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:41.957959 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:41.963233 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 75959.6us
I20260709 12:18:41.964960 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.996421 25783 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:41.966580 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.967403 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.894982 26335 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:41.939918 27936 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:41.947362 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:41.991026 25782 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:41.938941 28570 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:41.993568 27534 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:41.996799 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:42.005908 25704 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:42.010829 25779 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:41.989452 25787 raft_consensus.cc:2376] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6. Re-sending same reply.
I20260709 12:18:42.012452 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:41.998175 25784 raft_consensus.cc:2376] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7. Re-sending same reply.
I20260709 12:18:42.001586 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:42.000888 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2003) took 1061 ms. Trace:
W20260709 12:18:42.015810 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.017172 25780 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:41.976193 28623 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:42.015617 25634 rpcz_store.cc:276] 0709 12:18:40.939223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.537149 (+597926us) service_pool.cc:224] Handling call
0709 12:18:41.577765 (+ 40616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.577961 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:41.788592 (+210631us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.000553 (+211961us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:42.000610 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:42.000657 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.000721 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.000830 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161577,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":635,"thread_start_us":367774,"threads_started":2}
I20260709 12:18:42.059672 26344 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:42.063022 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.064286 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:42.065057 25785 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:42.067036 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.067788 25783 raft_consensus.cc:2376] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7. Re-sending same reply.
I20260709 12:18:41.994602 28419 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:41.960057 28603 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.097378 28603 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:42.072504 28570 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.103183 28570 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:42.073930 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.107574 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.061642 28623 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:42.074957 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.113175 26341 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:42.113476 25780 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:42.083424 27534 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:42.086630 28699 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.087396 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:42.090900 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:42.091887 28415 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:42.066061 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:42.096133 25787 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:42.096066 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.117136 25786 raft_consensus.cc:2368] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
W20260709 12:18:42.112279 26338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 676) took 2044 ms (client timeout 2040 ms). Trace:
I20260709 12:18:42.114645 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.069793 28290 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:42.117645 26338 rpcz_store.cc:269] 0709 12:18:40.068018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.085242 (+ 17224us) service_pool.cc:224] Handling call
0709 12:18:42.112104 (+2026862us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b540061c454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:42.112211 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27705088}
I20260709 12:18:42.062031 28519 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:41.975595 28595 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.127338 25786 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 12:18:42.127875 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:42.060137 28520 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:42.123484 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45075.7us
I20260709 12:18:42.121241 28699 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.126001 28290 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.059166 25778 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 3.
I20260709 12:18:42.130842 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.115936 28623 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:42.143512 25786 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:18:42.000778 28139 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.207367 27936 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.208570 28139 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:42.208729 27936 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:42.149978 25781 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:42.210937 25706 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:42.220345 28481 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.221730 25635 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:42.224846 28481 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:42.164907 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:42.227172 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62370.4us
I20260709 12:18:42.166983 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.168862 25713 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:42.169499 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.171109 28620 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:42.183034 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:42.240023 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 57112.9us
I20260709 12:18:42.176522 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:42.185329 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1656) took 1599 ms. Trace:
I20260709 12:18:42.142568 28519 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:42.191342 28290 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:42.194974 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:42.200323 28595 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:42.202152 28699 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:42.127269 28675 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:42.205724 28598 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.131645 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:42.225898 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:42.151445 27534 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:42.236367 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.244306 25632 rpcz_store.cc:276] 0709 12:18:40.585402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.094681 (+509279us) service_pool.cc:224] Handling call
0709 12:18:41.746875 (+652194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.747100 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:41.747435 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.818894 (+ 71459us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:41.818962 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:41.819008 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.819078 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.173349 (+354271us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b5400301bd0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:42.185267 (+ 11918us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10693,"prepare.queue_time_us":6440,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":14131,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":43471,"spinlock_wait_cycles":1475331840,"thread_start_us":60310,"threads_started":1,"wal-append.queue_time_us":64275}
I20260709 12:18:42.276276 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.278384 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:42.237205 28520 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:42.290787 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:42.291949 25783 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:42.294682 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:42.301914 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:18:42.289810 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 60587.4us
I20260709 12:18:42.295397 25709 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:42.303857 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.304473 25709 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
W20260709 12:18:42.297631 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61390.2us
I20260709 12:18:42.305403 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.269703 28139 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:42.269991 28570 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:42.325608 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:42.329222 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:42.332479 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:42.336783 26343 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:42.337421 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:42.337973 27534 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:42.338397 27534 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:42.338449 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.338826 27534 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:42.283743 28481 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:42.340583 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:42.342743 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:42.343215 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:42.333925 25603 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:42.303035 28617 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:42.347577 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 276269us
I20260709 12:18:42.267493 28603 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:18:42.323797 28583 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:42.337255 28593 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:42.361217 25707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.010s	sys 0.008s
I20260709 12:18:42.408792 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:42.411783 25707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.013s	sys 0.011s
I20260709 12:18:42.341667 28654 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:42.341600 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:42.424841 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.346268 28517 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:42.421309 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82946.4us
I20260709 12:18:42.445341 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:42.423470 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 82952.6us
W20260709 12:18:42.446349 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.447757 25676 leader_election.cc:400] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:42.448186 25676 leader_election.cc:403] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:18:42.419867 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.417273 28596 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.423386 28617 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:42.431764 28595 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.423190 28587 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:42.439213 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.439736 28652 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:42.440976 28593 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:42.450443 26308 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.452323 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.460700 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:42.461323 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.484025 25710 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:42.470788 28617 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.471962 28595 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.466992 28346 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:42.331255 28594 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 12:18:42.410400 28558 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:42.414844 28651 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:42.474095 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2223) took 1510 ms. Trace:
I20260709 12:18:42.450644 28699 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:42.348385 28570 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:42.343884 28139 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.359519 28580 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.503746 25708 rpcz_store.cc:276] 0709 12:18:40.963212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.742426 (+779214us) service_pool.cc:224] Handling call
0709 12:18:41.742813 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.993408 (+250595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.473800 (+480392us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400236254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:42.474025 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455786368}
I20260709 12:18:42.480103 28652 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:42.468237 28490 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:42.411194 28481 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.444218 28517 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.509100 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.420615 28519 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.475471 28597 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5.
I20260709 12:18:42.515079 25708 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:42.504592 25778 raft_consensus.cc:2393] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:42.546229 28711 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:18:42.550081 28346 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:42.551676 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:42.483870 27534 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.484571 28604 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:42.506806 28369 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:42.467469 28596 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.509858 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.578109 25712 raft_consensus.cc:2410] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:42.535952 25637 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:42.507375 28651 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:42.537914 28709 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:18:42.491835 28654 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:18:42.532788 28517 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:42.531133 28558 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:42.558743 28368 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.001s
I20260709 12:18:42.627354 28580 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:42.629361 28652 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.604965 28598 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:42.630890 28652 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:42.582177 28651 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:42.633797 25634 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:42.636844 25708 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:42.577363 28369 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:42.643798 25712 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:18:42.648706 25712 pending_rounds.cc:85] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 7
I20260709 12:18:42.649024 25712 pending_rounds.cc:107] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:18:42.649329 25712 raft_consensus.cc:2889] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:42.667410 25781 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:18:42.648075 28604 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:18:42.698805 25784 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:42.700758 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 607) took 1600 ms. Trace:
I20260709 12:18:42.704126 25638 rpcz_store.cc:276] 0709 12:18:41.100249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.746227 (+645978us) service_pool.cc:224] Handling call
0709 12:18:41.752529 (+  6302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.984389 (+231860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.700430 (+716041us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400363b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:42.700668 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512618368}
W20260709 12:18:42.702318 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.705583 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.706038 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.638525 28570 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:42.695305 28551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.339s	user 0.011s	sys 0.019s
I20260709 12:18:42.731582 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:42.736855 28551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.011s	sys 0.019s
I20260709 12:18:42.744431 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 739) took 2097 ms. Trace:
I20260709 12:18:42.744840 26341 rpcz_store.cc:276] 0709 12:18:40.646732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.890419 (+243687us) service_pool.cc:224] Handling call
0709 12:18:42.112995 (+1222576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.113528 (+   533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.744236 (+630708us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540061c450. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:42.744366 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2810871680}
I20260709 12:18:42.752380 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:42.753232 25638 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:42.741590 28558 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:42.747094 28595 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:42.719316 28369 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:42.764946 25712 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:42.758237 28558 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.769290 28558 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:42.769464 28139 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:42.770679 28139 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:42.759661 28587 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:42.793262 28369 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:42.798894 28596 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:42.802654 27936 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.004s
W20260709 12:18:42.815259 28719 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:18:42.755785 28481 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:42.831677 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.838104 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.818470 28604 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:42.842069 28623 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.000s
I20260709 12:18:42.849282 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:42.850898 26344 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:42.851888 26337 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:42.853457 26340 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:18:42.853868 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 393228us
I20260709 12:18:42.826387 28698 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:42.846403 28720 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:18:42.744565 28551 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:18:42.856500 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.857884 26341 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:18:42.857643 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 405351us
I20260709 12:18:42.864259 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:42.918511 26340 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:42.860790 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:42.923465 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 474998us
W20260709 12:18:42.923714 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 63022.2us
I20260709 12:18:42.924444 25751 leader_election.cc:400] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:18:42.925480 25751 leader_election.cc:400] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:42.927436 26344 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:42.926403 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:42.934993 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78576.8us
I20260709 12:18:42.935938 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:42.947844 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:42.852248 28574 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.954063 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.957376 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.964169 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 477079us
I20260709 12:18:42.964814 26340 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
W20260709 12:18:42.966400 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.966666 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45460us
I20260709 12:18:42.972512 25676 leader_election.cc:400] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9.
I20260709 12:18:42.972894 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.976500 28594 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:42.857993 28623 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.850983 28604 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:42.985029 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.963402 28290 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:42.966626 28551 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:42.969584 28596 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.000229 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 694) took 1157 ms. Trace:
I20260709 12:18:43.001729 28725 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:43.002676 28737 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:43.002617 28725 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.005793 28725 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.001978 28734 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:43.008507 28651 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.015055 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:42.969262 28574 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.018158 28574 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.019507 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.021446 28604 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:42.997851 28459 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:43.022480 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:42.959604 28649 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:18:42.972875 25676 leader_election.cc:403] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:43.026906 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:43.000720 26343 rpcz_store.cc:276] 0709 12:18:41.842788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.844639 (+  1851us) service_pool.cc:224] Handling call
0709 12:18:43.000001 (+1155362us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54000a2b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:43.000165 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841375488}
I20260709 12:18:43.035646 28398 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:43.036291 28398 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:43.038980 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1663) took 2132 ms. Trace:
W20260709 12:18:43.038875 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 66512.9us
I20260709 12:18:42.970202 27936 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:18:43.024124 28623 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:43.052956 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:43.039397 25630 rpcz_store.cc:276] 0709 12:18:40.914384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.252925 (+338541us) service_pool.cc:224] Handling call
0709 12:18:41.284928 (+ 32003us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:41.452026 (+167098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.452059 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:41.456842 (+  4783us) log.cc:844] Serialized 52 byte log entry
0709 12:18:42.452205 (+995363us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b1800376b40. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:18:42.824837 (+372632us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:42.824895 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.825015 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.024116 (+199101us) raft_consensus.cc:1685] finished
0709 12:18:43.024159 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.038905 (+ 14746us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179638,"raft.queue_time_us":160445,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":10090752,"thread_start_us":372350,"threads_started":1,"wal-append.queue_time_us":1528562}
I20260709 12:18:43.038581 28737 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:43.051400 28459 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.047169 28652 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9.
I20260709 12:18:43.093890 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:43.106966 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:43.114885 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1637) took 1038 ms. Trace:
W20260709 12:18:43.117861 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 367230us
W20260709 12:18:43.089732 28722 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.327s	user 0.001s	sys 0.001s
I20260709 12:18:43.120694 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.042057 28596 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.124562 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:43.125412 25676 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.594s (SENT)
I20260709 12:18:43.090054 28725 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.130000 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:43.037480 28398 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.130941 28604 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:43.025570 28594 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:43.136704 25676 leader_election.cc:336] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.713s (SENT)
I20260709 12:18:43.137120 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 612) took 1250 ms. Trace:
I20260709 12:18:43.137245 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:43.137988 25637 rpcz_store.cc:276] 0709 12:18:41.886795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.888505 (+  1710us) service_pool.cc:224] Handling call
0709 12:18:41.888924 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.584668 (+695744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.136764 (+552096us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b5400363654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:43.137040 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487350656}
I20260709 12:18:43.128935 25786 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:43.091619 28726 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:43.140314 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:43.141167 26340 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
W20260709 12:18:43.142127 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.147891 25676 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.814s (SENT)
I20260709 12:18:43.148473 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:43.148298 28675 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:43.150023 28726 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:43.149669 28675 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:43.151515 28726 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:43.127380 28651 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.171134 28617 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:43.195691 28728 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:43.198128 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2237) took 1013 ms. Trace:
I20260709 12:18:43.198711 25709 rpcz_store.cc:276] 0709 12:18:42.184564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.324098 (+139534us) service_pool.cc:224] Handling call
0709 12:18:42.346964 (+ 22866us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:42.425650 (+ 78686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.425686 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:42.426341 (+   655us) log.cc:844] Serialized 52 byte log entry
0709 12:18:42.759758 (+333417us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:42.759803 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.759890 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.197778 (+437888us) raft_consensus.cc:1685] finished
0709 12:18:43.197811 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.198041 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60431,"raft.queue_time_us":70488,"spinlock_wait_cycles":4368256,"thread_start_us":319607,"threads_started":1,"wal-append.queue_time_us":647306}
W20260709 12:18:43.207887 28710 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.782s	user 0.010s	sys 0.002s
I20260709 12:18:43.216373 28594 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:43.217674 28738 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:18:43.217814 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:43.219064 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.222150 28517 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.001s
I20260709 12:18:43.222882 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2100) took 1198 ms. Trace:
I20260709 12:18:43.223266 25782 rpcz_store.cc:276] 0709 12:18:42.024248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.244678 (+220430us) service_pool.cc:224] Handling call
0709 12:18:42.244998 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.297165 (+ 52167us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:42.339689 (+ 42524us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.543173 (+203484us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b1800359200. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a 00007f29b5999a2a
0709 12:18:42.916306 (+373133us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.960137 (+ 43831us) log.cc:844] Serialized 26 byte log entry
0709 12:18:42.960286 (+   149us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:42.960320 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.960407 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.222561 (+262154us) raft_consensus.cc:1685] finished
0709 12:18:43.222663 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.222816 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51203,"prepare.queue_time_us":42451,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":3047,"raft.queue_time_us":53872,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":36959,"spinlock_wait_cycles":3983616,"thread_start_us":343416,"threads_started":1,"wal-append.queue_time_us":731876}
I20260709 12:18:43.225615 27936 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.226257 27936 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:43.226964 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2163) took 1775 ms (client timeout 1950 ms). Trace:
W20260709 12:18:43.227346 25706 rpcz_store.cc:269] 0709 12:18:41.451341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.451468 (+   127us) service_pool.cc:224] Handling call
0709 12:18:43.226732 (+1775264us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54002ec7d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:43.226899 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1744168832}
I20260709 12:18:43.228396 27936 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:43.229066 25711 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:43.139810 28587 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:43.231225 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 380) took 1650 ms. Trace:
I20260709 12:18:43.231587 25711 rpcz_store.cc:276] 0709 12:18:41.581263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.582705 (+  1442us) service_pool.cc:224] Handling call
0709 12:18:43.228889 (+1646184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.229479 (+   590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.231030 (+  1551us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54002ec7d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:43.231165 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3785042816}
I20260709 12:18:43.232766 28596 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:43.146967 28610 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:43.234593 28398 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:43.234967 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.236009 25637 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:43.163617 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.247565 26343 raft_consensus.cc:2410] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:43.260918 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 609) took 1800 ms. Trace:
I20260709 12:18:43.260728 28517 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.251327 28594 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:43.261652 25631 rpcz_store.cc:276] 0709 12:18:41.460171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.751009 (+290838us) service_pool.cc:224] Handling call
0709 12:18:41.751291 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:41.755432 (+  4141us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:41.768950 (+ 13518us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.995148 (+226198us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.056989 (+ 61841us) log.cc:844] Serialized 26 byte log entry
0709 12:18:42.057156 (+   167us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:42.057194 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.057273 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.207041 (+1149768us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:43.260437 (+ 53396us) raft_consensus.cc:1685] finished
0709 12:18:43.260605 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.260831 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9356,"prepare.queue_time_us":13177,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":6314,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":7849,"spinlock_wait_cycles":40945408,"thread_start_us":188977,"threads_started":1,"wal-append.queue_time_us":519672}
I20260709 12:18:43.263326 28594 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.264737 25636 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:43.158203 28721 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 12:18:43.269230 25750 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:43.141973 28089 raft_consensus.cc:2764] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:18:43.117547 28459 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.277209 27534 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:43.279280 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 315613us
I20260709 12:18:43.251417 28649 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.139504 28725 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.279911 25631 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:43.265017 27936 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.284593 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.276336 28625 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.281136 28570 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:43.289434 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.291960 28604 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:43.295047 28625 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:43.307410 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.308718 28749 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:43.295406 28651 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.313843 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.315646 28517 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.316824 28594 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.330610 28623 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.331243 28623 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.333459 28623 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:43.318266 28739 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:18:43.336604 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 453) took 1075 ms. Trace:
I20260709 12:18:43.337024 25778 rpcz_store.cc:276] 0709 12:18:42.261607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.545855 (+284248us) service_pool.cc:224] Handling call
0709 12:18:42.546530 (+   675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.547172 (+   642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.547219 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.547871 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 12:18:42.951753 (+403882us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:42.964475 (+ 12722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.973200 (+  8725us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.336372 (+363172us) raft_consensus.cc:1685] finished
0709 12:18:43.336391 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.336541 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9768,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5502,"thread_start_us":367210,"threads_started":1,"wal-append.queue_time_us":565345}
I20260709 12:18:43.339712 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2097) took 1320 ms. Trace:
I20260709 12:18:43.289381 28649 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.318599 28733 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:43.340114 25780 rpcz_store.cc:276] 0709 12:18:42.019486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.203942 (+184456us) service_pool.cc:224] Handling call
0709 12:18:42.204288 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.301851 (+ 97563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.301884 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:42.301946 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:42.301983 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.339488 (+1037505us) spinlock_profiling.cc:243] Waited 97.3 ms on lock 0x7b54002b4654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:43.339538 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.339654 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223861120}
I20260709 12:18:43.345801 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.348416 28607 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:43.349233 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:43.350668 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:43.351357 26337 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:43.292863 27936 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:43.355939 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 408158us
I20260709 12:18:43.355074 28517 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.362043 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:43.362993 26341 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:43.362322 28517 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:43.366974 28743 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:18:43.367672 28517 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.340540 28736 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.340740 28368 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.368304 28745 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:43.351758 28594 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:43.321967 28651 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.361137 28596 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:43.361322 28604 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.385216 28604 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.387495 28604 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.383955 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:18:43.380427 28558 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.000s
I20260709 12:18:43.376459 28139 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:43.395520 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.394714 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.396003 26343 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:43.394099 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 268976us
I20260709 12:18:43.423821 28558 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.426808 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:43.427580 25707 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:43.429692 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.439179 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.439926 26338 raft_consensus.cc:2376] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7. Re-sending same reply.
W20260709 12:18:43.440649 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 46010.2us
I20260709 12:18:43.395272 28559 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:43.442119 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:43.439677 28570 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:43.428658 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.445914 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:43.453698 25708 raft_consensus.cc:2410] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:43.450340 25706 pending_rounds.cc:85] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 2
I20260709 12:18:43.454324 25706 pending_rounds.cc:107] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:18:43.454857 25706 raft_consensus.cc:2889] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.377512 28368 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.378257 28736 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.455804 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:43.422427 28594 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.422302 28609 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:43.458238 25707 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:43.420387 28139 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:43.468922 26344 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:43.428308 28558 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:43.473815 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:43.475574 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:43.478586 25709 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:18:43.479013 25709 pending_rounds.cc:85] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 6
I20260709 12:18:43.479244 25709 pending_rounds.cc:107] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:18:43.479264 28649 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:43.479624 25709 raft_consensus.cc:2889] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.480690 27936 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:43.480943 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:43.472707 28594 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.482965 28610 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:43.483698 28368 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:43.486032 28746 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:18:43.481666 25709 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:18:43.476778 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:43.447541 28459 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:43.470263 28725 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.441848 28749 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:18:43.452261 28559 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:43.489609 28369 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.006s
I20260709 12:18:43.473472 28139 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:43.471385 28748 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:43.504920 25704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 383) took 1922 ms (client timeout 1880 ms). Trace:
W20260709 12:18:43.518716 25704 rpcz_store.cc:269] 0709 12:18:41.582296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.582462 (+   166us) service_pool.cc:224] Handling call
0709 12:18:43.504687 (+1922225us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400147154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:43.504847 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864008960}
I20260709 12:18:43.506196 25713 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:43.458664 28587 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:18:43.530515 28757 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.492671 28551 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:43.498090 28570 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.544278 28558 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.504215 28559 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:43.518679 28748 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.487888 28609 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.571897 28681 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.582957 28681 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.586161 28651 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.587198 28681 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.587328 28594 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:43.587682 28748 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:43.587777 28736 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:43.504331 28732 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
W20260709 12:18:43.620096 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 168504us
W20260709 12:18:43.624850 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 90306.3us
I20260709 12:18:43.625506 25633 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:43.625502 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 165057us
I20260709 12:18:43.557839 28558 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.554643 28570 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.648726 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:43.649752 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.652135 25781 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:43.651492 26340 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:43.651326 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:43.552155 28757 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.674387 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1635) took 1604 ms. Trace:
I20260709 12:18:43.649400 28749 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.635515 28440 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:18:43.661958 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.664058 28736 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:43.664336 28558 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:43.672829 28609 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:43.674832 25779 rpcz_store.cc:276] 0709 12:18:42.069653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.212752 (+1143099us) service_pool.cc:224] Handling call
0709 12:18:43.213317 (+   565us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:43.291011 (+ 77694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.291062 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:43.294686 (+  3624us) log.cc:844] Serialized 52 byte log entry
0709 12:18:43.471849 (+177163us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:43.497930 (+ 26081us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.502175 (+  4245us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.542689 (+ 40514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.673523 (+130834us) spinlock_profiling.cc:243] Waited 9.79 ms on lock 0x7b18000dd800. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419
0709 12:18:43.674048 (+   525us) spinlock_profiling.cc:243] Waited 72.7 ms on lock 0x7b54000b04d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:43.674113 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.674157 (+    44us) raft_consensus.cc:1685] finished
0709 12:18:43.674174 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.674322 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8082,"prepare.queue_time_us":68799,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2784,"raft.queue_time_us":90893,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":8412,"spinlock_wait_cycles":234591360,"thread_start_us":118870,"threads_started":1,"wal-append.queue_time_us":240789}
I20260709 12:18:43.683054 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:43.720351 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.721065 25631 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
W20260709 12:18:43.715788 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.682632 25635 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:43.716737 25637 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:43.736402 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.737416 25632 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:43.738379 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 87186.4us
I20260709 12:18:43.739243 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:43.742136 25634 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:18:43.744861 25634 pending_rounds.cc:85] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 6
I20260709 12:18:43.745162 25634 pending_rounds.cc:107] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:18:43.745428 25634 raft_consensus.cc:2889] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.747846 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.748682 25629 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:43.675047 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:43.751070 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:43.752166 25782 raft_consensus.cc:2376] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5. Re-sending same reply.
W20260709 12:18:43.752661 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 303031us
I20260709 12:18:43.759470 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.713338 28440 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:43.731802 28369 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.771297 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:43.775952 26336 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:43.773324 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:43.777160 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.773780 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:43.774430 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 140099us
I20260709 12:18:43.778868 26339 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:43.780548 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.780921 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.781563 25779 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:43.782997 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:43.753737 28609 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:43.786276 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.788944 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.791157 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.792466 25635 raft_consensus.cc:2410] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:43.791216 25631 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:43.793430 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.794440 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.678799 28766 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 12:18:43.722028 28749 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.797139 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.797191 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:43.797919 25782 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:43.793401 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.801759 28481 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:43.801338 28440 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.802205 28481 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.802717 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:43.803857 28368 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:18:43.805537 28768 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:43.806577 25778 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:43.799393 25635 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:43.775259 28757 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:43.731875 28736 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:43.819519 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 80362.8us
I20260709 12:18:43.820410 25784 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:18:43.814442 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:43.820195 25638 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:43.822131 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.802670 28481 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:43.787034 28517 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:43.789846 28623 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.827687 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:43.852931 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 57406us
I20260709 12:18:43.863662 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:43.864737 26344 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:43.786264 28369 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.900655 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.826184 28751 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.863219 28551 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:43.917222 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.866109 28762 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:43.866415 28767 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.846217 28757 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
W20260709 12:18:43.933867 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 115938us
W20260709 12:18:43.929595 28726 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.008s
I20260709 12:18:43.900213 28651 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:43.851397 28517 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.864063 28623 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:43.936930 28551 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.942011 28751 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:43.947359 28772 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:43.952086 28725 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:43.957521 28596 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.956250 28651 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:43.997327 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:43.997933 28609 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:43.997272 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:43.964871 28551 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.998884 28609 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.023347 25778 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:43.951831 28726 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:44.046584 28775 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:18:44.047955 28751 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.048950 28736 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.026947 28597 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:44.051680 28597 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:44.022830 28783 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:44.042253 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:44.051381 28139 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 12:18:44.055578 28603 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:18:43.953087 28767 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:44.062002 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.057478 28139 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.021816 28596 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.072109 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.072773 25784 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.073316 28596 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.041523 28725 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:44.076284 28767 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.077248 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:44.078326 28625 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.758s	user 0.013s	sys 0.009s
I20260709 12:18:44.078814 28625 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.080966 28749 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.081197 25784 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:44.081647 28625 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.082736 28598 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:44.082634 28725 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.083253 28598 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:44.084169 28598 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.085062 28725 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.085209 28598 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:44.086831 28623 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:44.079456 28774 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:18:44.060681 28783 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.061766 28751 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.116552 25786 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:44.118068 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.061209 28776 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:44.068434 28570 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.123181 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.124637 25780 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:18:44.124967 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.130055 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.135854 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.136689 25787 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:44.057739 28651 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.139762 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.140412 28651 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.141711 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.143205 28369 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.143991 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.144380 28783 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.144667 25785 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
W20260709 12:18:44.147079 28559 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260709 12:18:44.147506 28559 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.149653 28726 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.011380 28772 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.151409 25779 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:44.149712 28559 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.132459 28776 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.158890 28796 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:44.159500 28780 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.159797 28796 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.160055 28780 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.160301 28796 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.166828 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.168215 28658 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:44.168799 28658 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.171056 28658 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.171604 28658 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:44.172116 28658 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.172609 28517 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.175364 28658 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.177954 28625 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.182726 28625 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.183321 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.184213 26344 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:44.186394 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:44.186977 28625 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.177546 28773 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.069180 28757 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:18:44.187920 28773 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.178359 28796 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.178833 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.180606 28772 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.056810 28597 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.125655 28751 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:44.118918 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 214886us
I20260709 12:18:44.190449 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2009) took 2104 ms. Trace:
I20260709 12:18:44.197417 25636 rpcz_store.cc:276] 0709 12:18:42.091468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.351259 (+259791us) service_pool.cc:224] Handling call
0709 12:18:42.351522 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.269123 (+917601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.190218 (+921095us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400371ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:44.190390 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099626752}
W20260709 12:18:44.195045 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.196658 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.143215 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.193449 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2234) took 2035 ms. Trace:
I20260709 12:18:44.200537 25780 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:44.201335 25713 rpcz_store.cc:276] 0709 12:18:42.157825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.238129 (+ 80304us) service_pool.cc:224] Handling call
0709 12:18:43.506065 (+1267936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.538055 (+ 31990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.193267 (+655212us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400147150. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:44.193390 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2847164800}
I20260709 12:18:44.200460 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:44.203914 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 206702us
I20260709 12:18:44.195922 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:44.208148 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 165943us
I20260709 12:18:44.209887 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:44.209815 28787 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:44.212248 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:44.213383 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:44.214794 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:44.159000 28776 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:44.192286 28752 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.630s	user 0.000s	sys 0.001s
I20260709 12:18:44.207687 28517 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.242477 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.192986 28597 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:44.242240 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.243611 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:44.250818 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:44.251772 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:44.252035 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.247869 25785 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:44.253360 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.244920 25782 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:44.254070 25781 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:44.256032 25750 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.196378 28780 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.286005 26337 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:44.286085 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:44.289183 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:44.258958 26344 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:44.193809 28773 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.291718 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.295701 26337 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:44.296689 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:44.263386 28748 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.000s
I20260709 12:18:44.213596 28625 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:44.334640 25676 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.019s	sys 0.029s
I20260709 12:18:44.242307 28623 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:44.342208 26336 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:44.297514 26307 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:44.344965 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47535.1us
W20260709 12:18:44.337103 25676 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.019s	sys 0.029s
W20260709 12:18:44.345835 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 89889.5us
I20260709 12:18:44.281418 28649 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:44.258771 28517 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.253713 28415 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:44.286197 28725 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:44.285637 28730 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:44.299052 28757 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:44.389310 28757 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.389770 28757 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.384871 28788 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:44.397148 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:44.398746 28617 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:44.399036 28139 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.399200 28609 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.399654 28609 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.400013 28609 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.403080 28658 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.403621 28623 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:44.403702 28658 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.406002 28730 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:44.383587 28773 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.406793 28749 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:44.406877 28773 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:44.407517 28773 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.406150 28658 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.294250 28751 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:44.344062 28733 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.006s	sys 0.021s
W20260709 12:18:44.347256 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 523154us
W20260709 12:18:44.412541 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.413178 25676 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 2.198s (SENT)
I20260709 12:18:44.339272 28598 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:44.414834 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.415225 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:44.418627 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 129544us
I20260709 12:18:44.339707 28625 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:44.419173 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 168620us
I20260709 12:18:44.350194 28772 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:44.292038 28789 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.174s
I20260709 12:18:44.451333 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:44.414153 28733 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.006s	sys 0.022s
I20260709 12:18:44.436959 26342 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:44.446919 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.459462 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 706) took 1020 ms. Trace:
I20260709 12:18:44.459780 28725 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.411149 28762 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:44.411782 28649 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:44.448617 28790 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.451277 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:44.345816 28594 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:18:44.411721 28517 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:44.477979 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.337052 28748 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.416734 28658 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:18:44.460327 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.459862 26342 rpcz_store.cc:276] 0709 12:18:43.439128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.439273 (+   145us) service_pool.cc:224] Handling call
0709 12:18:44.422270 (+982997us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:44.422461 (+   191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.459207 (+ 36746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.459408 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13913856}
W20260709 12:18:44.470664 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57490.2us
I20260709 12:18:44.448861 28625 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.424688 28598 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.396502 28551 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.456508 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:44.384013 28481 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.505355 28481 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:44.505985 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49575.5us
W20260709 12:18:44.484961 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 2.200s (SENT) [suppressed 50 similar messages]
W20260709 12:18:44.507112 28794 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:18:44.508504 28802 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:44.453740 28780 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:44.487846 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:44.512040 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.512224 28814 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:44.512881 28481 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:44.513038 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:44.513345 28481 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.513849 28481 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.514765 28806 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:44.512250 25634 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:44.522063 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.523017 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.523707 25784 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
W20260709 12:18:44.488754 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.469708 28788 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.462278 28734 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.461009 28725 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.493759 28598 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:44.532191 28734 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.535166 28734 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.529562 28814 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:44.420315 28730 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.538818 28725 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.540215 28730 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.502970 28551 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.527346 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.542570 25635 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:44.543680 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.471824 28415 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:18:44.544289 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.544714 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:44.545320 25635 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:44.527670 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.546334 25637 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:44.528028 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.497483 28698 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:44.518788 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.494055 28369 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.470362 28772 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.547860 25782 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:44.477967 28790 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:44.579715 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.551697 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:44.552747 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.559935 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:44.560429 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.563892 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.587685 26340 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:44.591074 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.591666 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:44.570389 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 161161us
I20260709 12:18:44.578579 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.593088 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.593952 26341 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:44.594267 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.546391 28649 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.534251 28658 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:18:44.600534 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:44.601300 26336 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.598111 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:44.598901 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.602633 26340 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:44.559314 28698 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:44.597682 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.582828 28790 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.604429 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.612938 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.613771 25780 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:44.607120 27936 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.615890 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.612702 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.609632 25631 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:44.597306 28459 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:44.586992 28795 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:44.619656 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.598203 28139 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:44.603315 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.548337 28788 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.603773 28796 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:44.631443 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:44.630023 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 119636us
I20260709 12:18:44.611982 28698 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:44.581832 28369 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.615646 27936 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.620505 28559 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:44.603577 28770 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:18:44.639950 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:44.646296 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:44.646862 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 140177us
I20260709 12:18:44.625861 28795 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.623868 28811 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:44.658368 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.650985 28559 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.634629 28813 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:44.648306 27936 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.653851 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:44.654698 28795 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.669106 28559 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.677000 28747 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:44.653429 28597 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:44.678025 28817 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:44.679869 28788 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.681332 25635 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:44.790311 25635 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:44.790737 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:44.791955 25635 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:44.684374 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:44.651503 28791 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:44.689446 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:44.796068 25638 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:43.115384 25783 rpcz_store.cc:276] 0709 12:18:42.075979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.300971 (+224992us) service_pool.cc:224] Handling call
0709 12:18:42.327818 (+ 26847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.328053 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:42.745600 (+417547us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.114380 (+368780us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.114571 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:43.114617 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:43.114657 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.114720 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.114830 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11591,"prepare.queue_time_us":108508,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":751,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":16886,"thread_start_us":729352,"threads_started":2}
I20260709 12:18:44.799466 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.800170 25783 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:44.690986 28807 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:18:44.737059 28517 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.656035 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:44.697801 28598 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.652168 28796 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:44.741142 28609 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.744228 28734 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:44.743883 28779 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.773s	user 0.005s	sys 0.004s
I20260709 12:18:44.734757 28806 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:44.666422 28783 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.795622 28791 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.673864 28792 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.749751 25779 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:18:44.751291 26335 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:44.753149 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:44.673965 28813 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:44.737671 28772 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:44.645660 28139 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:44.768842 25786 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:44.779918 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:44.780750 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 127008us
I20260709 12:18:44.781129 25787 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:44.783568 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:44.840430 25630 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.667732 28459 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:44.777138 26336 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
W20260709 12:18:44.760607 28651 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.000s
I20260709 12:18:44.810928 25638 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:18:44.811884 26308 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:44.812954 27936 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.786448 28748 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.788218 28817 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:44.658682 25780 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:18:44.831740 25783 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:44.795598 28820 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.834074 25629 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:44.887805 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:44.888425 26342 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:44.836410 25632 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:18:44.841953 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 108084us
I20260709 12:18:44.844883 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:44.891464 25705 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:18:44.891537 27936 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.845610 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:44.892411 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:44.835031 26337 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:44.893185 28598 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.893453 28139 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.893922 28570 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:44.894405 28570 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:44.847622 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:44.866726 28806 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.895654 28806 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.895290 28570 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.896071 28806 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.896385 28570 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:44.897737 25534 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 193) took 16604 ms (client timeout 15000 ms). Trace:
I20260709 12:18:44.814193 28805 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:18:41.831934 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:44.836684 28791 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:44.899884 28609 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.873744 28821 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:44.900626 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 532) took 1553 ms. Trace:
I20260709 12:18:44.900698 28821 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:44.901160 26343 rpcz_store.cc:276] 0709 12:18:43.347720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.347849 (+   129us) service_pool.cc:224] Handling call
0709 12:18:43.348200 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.430804 (+ 82604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.900419 (+1469615us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b540044b454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:44.900574 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109382016}
I20260709 12:18:44.901650 28734 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.872854 28822 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:18:44.831305 28783 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.902621 28783 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.850854 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:44.878662 28796 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:44.846681 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:44.905284 28796 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:44.845012 28792 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.905716 28796 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:44.906878 28796 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.863654 28459 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.857146 28732 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:44.907855 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:44.898101 25534 rpcz_store.cc:269] 0709 12:18:28.293617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.293793 (+   176us) service_pool.cc:224] Handling call
0709 12:18:44.897594 (+16603801us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:35.674055 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:35.776044 (+101989us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:35.776146 (+   102us) write_op.cc:435] Acquired schema lock
0709 12:18:35.776185 (+    39us) tablet.cc:662] Decoding operations
0709 12:18:35.802959 (+ 26774us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:35.803110 (+   151us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:35.803182 (+    72us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:18:35.805083 (+  1901us) tablet.cc:701] Row locks acquired
0709 12:18:35.805152 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:18:35.805467 (+   315us) write_op.cc:270] Start()
0709 12:18:35.829482 (+ 24015us) write_op.cc:276] Timestamp: P: 1783599515805356 usec, L: 0
0709 12:18:35.829518 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:18:35.954527 (+125009us) log.cc:844] Serialized 5604 byte log entry
0709 12:18:38.290177 (+2335650us) op_driver.cc:464] REPLICATION: finished
0709 12:18:38.781060 (+490883us) write_op.cc:301] APPLY: starting
0709 12:18:38.792851 (+ 11791us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:38.805280 (+ 12429us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:38.806255 (+   975us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:38.808064 (+  1809us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:38.808493 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:18:38.808552 (+    59us) write_op.cc:312] APPLY: finished
0709 12:18:38.809076 (+   524us) log.cc:844] Serialized 101 byte log entry
0709 12:18:38.924348 (+115272us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:38.933119 (+  8771us) write_op.cc:454] Released schema lock
0709 12:18:38.958894 (+ 25775us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:38.984735 (+ 25841us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:38.984780 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4436590,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":474945,"apply.run_cpu_time_us":34232,"apply.run_wall_time_us":211403,"num_ops":14,"prepare.queue_time_us":861768,"prepare.run_cpu_time_us":57767,"prepare.run_wall_time_us":1101774,"raft.queue_time_us":1045284,"raft.run_cpu_time_us":7395,"raft.run_wall_time_us":404332,"replication_time_us":2406961,"spinlock_wait_cycles":6400,"thread_start_us":2249436,"threads_started":5,"wal-append.queue_time_us":1053993,"wal-append.run_cpu_time_us":28671,"wal-append.run_wall_time_us":2760675}]]}
I20260709 12:18:44.908504 28732 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:44.901780 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:44.909440 28574 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:44.910175 28574 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:44.908694 25785 raft_consensus.cc:2368] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7.
I20260709 12:18:44.912612 25752 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:44.913888 27936 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:44.914295 27936 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:44.914654 27936 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:44.915509 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 470) took 1098 ms. Trace:
I20260709 12:18:44.915993 25778 rpcz_store.cc:276] 0709 12:18:43.817132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.819252 (+  2120us) service_pool.cc:224] Handling call
0709 12:18:44.915308 (+1096056us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b5400380254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:44.915439 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189394688}
I20260709 12:18:44.901875 28821 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.918248 28772 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:44.846037 28813 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:44.918835 28772 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.884043 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:44.921836 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.922140 25711 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:44.922503 25780 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.923942 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.822714 28757 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.899240 28551 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:44.924490 25783 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:44.924862 28551 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:44.925828 28551 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.927387 28596 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:44.927874 28596 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:44.888475 25710 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:44.789705 28747 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.904922 28725 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:44.932096 28815 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:44.932592 28815 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:44.933421 28815 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.934921 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:44.935583 25632 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.936863 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:44.938781 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 52901.9us
I20260709 12:18:44.946532 26337 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:18:44.948033 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.957571 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45057.8us
I20260709 12:18:44.958360 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:44.907946 28796 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:44.961251 25787 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:44.963560 25778 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:18:44.964488 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.965087 25787 raft_consensus.cc:2376] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8. Re-sending same reply.
I20260709 12:18:44.965628 25630 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:44.968438 26343 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:18:44.909531 28732 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:44.972244 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:44.974934 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:44.984935 27534 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:44.976660 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.926234 28757 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:44.988332 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:44.990662 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.990907 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.995695 28789 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:44.998685 28089 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:45.001561 28793 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.002156 28793 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.005296 28793 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:44.968160 28440 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:45.008116 28369 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:45.008788 28815 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:44.898979 25535 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:45.010813 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:45.011409 25634 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:45.022291 26336 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:45.002920 25632 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:45.028720 25780 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:45.012651 25783 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:45.034178 27936 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.031749 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:45.035985 25779 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:45.026221 28651 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.040833 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:45.041066 28651 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.041643 25629 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:45.042795 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:44.981257 28767 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:45.045600 28559 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.048053 28587 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:45.048645 28587 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.011129 28725 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:45.010409 28763 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:45.051530 28763 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.056035 25631 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 12:18:45.056697 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 114069us
I20260709 12:18:45.057600 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:45.058101 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.067552 25710 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:18:44.968861 28751 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:44.969451 28773 consensus_meta.cc:287] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.561s	user 0.006s	sys 0.009s
W20260709 12:18:45.083340 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.080120 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53071.1us
I20260709 12:18:44.989982 28597 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.086072 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:45.094473 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 62816.8us
I20260709 12:18:44.997952 28762 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:18:45.095296 25601 leader_election.cc:400] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7.
I20260709 12:18:44.991997 28780 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:45.095597 25601 leader_election.cc:403] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:45.092633 25787 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:44.981559 28609 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:45.110435 28744 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:45.111466 28811 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:45.112005 28811 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:45.114171 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:45.055259 28517 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:45.100924 28763 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:45.104254 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:45.105402 28825 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:45.106805 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.106335 28822 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:45.108417 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:45.054953 28734 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.097782 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:44.968071 28781 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:18:45.121661 25634 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:45.078626 28814 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:45.125859 27936 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:45.099426 28773 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:45.187862 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 83694.7us
I20260709 12:18:45.191601 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:45.112972 28821 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:45.107828 28730 raft_consensus.cc:2764] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:45.112864 28747 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:45.114030 28819 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:45.026120 28596 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.196699 28805 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:45.118471 28369 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:45.194340 28776 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.912s	user 0.014s	sys 0.002s
I20260709 12:18:45.192858 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:45.223481 25780 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:45.192881 28519 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:45.194111 28725 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7.
I20260709 12:18:45.109560 28597 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.187824 28763 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:45.229192 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:45.239595 25710 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:45.230629 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.232954 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.117259 28780 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.118628 28609 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.245265 25636 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:45.202126 28596 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:45.213637 28757 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:45.263334 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:45.249683 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.218307 28822 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:45.274370 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.278637 25707 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:45.275236 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.279838 25711 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:45.226851 28519 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.294358 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.196985 28734 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.290460 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:45.305153 25711 raft_consensus.cc:2368] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:18:45.299604 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.235194 28819 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:45.302429 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 305927us
I20260709 12:18:45.273073 25535 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:45.310218 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.244751 28587 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:45.315075 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.319617 25535 catalog_manager.cc:5697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:45.245674 28805 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:45.324824 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.265672 28652 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:45.272619 28762 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.225999 28369 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.332026 25705 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:45.333177 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.277060 28757 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:45.301802 28519 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:45.340598 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.340785 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.372887 25704 raft_consensus.cc:2410] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:45.373060 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.229359 28776 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.375223 25707 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:45.376004 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:45.380918 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 238533us
I20260709 12:18:45.381371 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:45.339087 28790 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 12:18:45.254971 28763 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:45.339625 28625 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.814s	user 0.007s	sys 0.007s
W20260709 12:18:45.339398 28481 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.002s
I20260709 12:18:45.242771 28795 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.385430 25713 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:45.385342 25706 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:45.400704 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.401376 25707 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:45.373404 28757 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:45.392213 25535 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 3, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:45.410928 25780 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:45.413602 25710 raft_consensus.cc:2410] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:45.414804 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.302305 28726 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.395339 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:45.385046 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.418926 25704 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:45.394011 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:45.416430 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.382516 28818 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.375305 28815 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.374867 28820 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:45.412282 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:45.416481 25786 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:45.424197 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.398470 28419 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:45.429659 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.426549 25603 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:45.437400 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.372743 28652 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.471385 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:45.472370 25780 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:45.473835 28603 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.474431 28603 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.416217 28559 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:45.476061 28419 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.478111 28603 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:45.480258 28652 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:45.482867 28419 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:45.486384 28419 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
W20260709 12:18:45.501502 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 207514us
I20260709 12:18:45.512400 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:45.514256 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.394297 28781 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:18:45.372571 28762 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.435870 25704 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:45.522782 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.523216 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.523751 25780 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:45.524240 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:45.525229 25705 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:45.524384 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:45.529846 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.530777 25705 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:45.532407 25705 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:45.207993 28598 consensus_queue.cc:1175] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "955233c139344131a847044f17f8d348" responder_term: 5 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:18:45.553685 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.561735 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:45.492642 28748 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:45.554073 25704 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:45.555315 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:45.556716 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:45.554324 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:45.406265 28625 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.563916 25629 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:45.405942 28481 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.563225 25783 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:45.397157 28790 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.504419 28847 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:45.570039 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.566840 25712 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:45.468725 28649 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:18:45.425516 28726 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.468259 28815 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.546399 28652 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:18:45.440567 28818 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.439736 28819 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.439935 28822 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.509217 28805 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:45.625840 25782 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:18:45.909957 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.644465 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:45.616700 26338 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:45.584595 25535 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 3 to 4, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:45.596683 25784 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:46.302393 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2131) took 1380 ms. Trace:
W20260709 12:18:46.091238 25711 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.002s	sys 0.021s
W20260709 12:18:45.879926 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 439793us
W20260709 12:18:46.367360 25711 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.011s	sys 0.022s
W20260709 12:18:45.671353 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 421639us
I20260709 12:18:46.303978 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.369558 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.370430 25708 raft_consensus.cc:2410] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:46.369890 25535 catalog_manager.cc:5697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 2, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 2 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:46.371431 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.370168 28825 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:46.372923 28598 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:46.374763 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.471527 25708 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:46.306346 28776 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:46.381155 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.023s	sys 0.042s
I20260709 12:18:46.144665 28772 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:46.358331 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:46.450989 28762 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.482396 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:46.384052 25706 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.983s	user 0.001s	sys 0.031s
I20260709 12:18:45.815198 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.484892 25713 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:45.554375 28846 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:46.407764 25631 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:46.409989 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.387776 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.468564 28419 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.562623 28773 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:45.910315 28440 consensus_meta.cc:287] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.893s	user 0.010s	sys 0.002s
W20260709 12:18:45.629010 28831 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
W20260709 12:18:45.648401 28832 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
W20260709 12:18:45.810454 28836 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:18:46.386523 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:46.519318 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 132971us
W20260709 12:18:46.488813 28139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.105s	user 0.006s	sys 0.007s
W20260709 12:18:45.629925 28830 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 12:18:46.430521 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2195) took 1198 ms. Trace:
I20260709 12:18:46.531770 25712 rpcz_store.cc:276] 0709 12:18:45.231937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.232073 (+   136us) service_pool.cc:224] Handling call
0709 12:18:46.430282 (+1198209us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400066b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:46.430425 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624694016}
I20260709 12:18:46.431407 25710 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:46.105052 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1646) took 1142 ms. Trace:
I20260709 12:18:46.533519 25786 rpcz_store.cc:276] 0709 12:18:44.962743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.962886 (+   143us) service_pool.cc:224] Handling call
0709 12:18:44.963164 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.449893 (+486729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.104694 (+654801us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54002b5554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:46.104959 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042066816}
I20260709 12:18:46.029976 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:46.536082 25707 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:45.904731 28796 consensus_meta.cc:287] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.910s	user 0.011s	sys 0.000s
I20260709 12:18:45.799525 28369 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.384606 28551 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:46.522998 28139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.007s	sys 0.007s
I20260709 12:18:46.355940 28793 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:45.751664 28651 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:45.678304 28780 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:45.809648 28795 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:45.866667 28808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.011s	sys 0.032s
I20260709 12:18:46.031958 28346 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:46.510526 28419 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:45.754150 28792 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:45.816443 28459 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:45.617468 28847 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:45.615792 28820 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.843s
W20260709 12:18:46.063235 28791 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.158s	user 0.006s	sys 0.008s
W20260709 12:18:46.093874 28698 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.433s	user 0.010s	sys 0.007s
I20260709 12:18:46.563642 28346 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.563877 28791 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.564129 28698 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.553570 28793 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:46.565507 28793 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.384259 28763 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:45.985677 28813 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.066s	user 0.013s	sys 0.001s
I20260709 12:18:46.411217 28559 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:45.801534 28734 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:46.450173 28089 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.000s	sys 0.009s
I20260709 12:18:46.225693 28814 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:18:46.570137 28089 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.000s	sys 0.010s
I20260709 12:18:46.570272 28814 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:46.440637 28770 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.028s	sys 0.027s
I20260709 12:18:45.880769 28789 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:46.350953 28817 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.574925 28789 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.469338 28623 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:46.572304 28770 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.028s	sys 0.028s
I20260709 12:18:46.468804 28730 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:45.817909 28574 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:46.515072 27534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.044s	sys 0.087s
I20260709 12:18:46.517378 28609 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.568735 28791 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:45.817700 28596 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.000s
I20260709 12:18:45.630198 28597 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.480199 28858 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:18:46.479801 28598 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.499094 28481 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.474217 28825 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.543881 28773 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:46.502741 28857 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:46.609961 28773 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:46.610136 28857 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.611510 28773 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.613386 28857 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.542716 28790 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.541109 28736 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.614861 28736 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.537317 28846 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:45.581458 28748 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:46.618103 28748 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:46.557920 28808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.311s	user 0.023s	sys 0.039s
I20260709 12:18:46.554105 28625 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.566310 28698 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.570935 28815 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.575542 28817 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:46.554878 28570 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.658s	user 0.016s	sys 0.000s
I20260709 12:18:46.616338 28846 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.559707 28869 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:46.623024 28869 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.469008 28788 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.572505 28793 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:46.600571 28757 consensus_meta.cc:287] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.177s	user 0.006s	sys 0.005s
I20260709 12:18:46.571192 28813 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.632877 28788 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.634846 28813 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:46.635797 28788 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.572543 28559 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.757978 28559 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.455370 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.842945 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.843567 25630 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:46.844270 25634 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:46.455062 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.846274 25633 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
W20260709 12:18:45.854070 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.015s	sys 0.022s
W20260709 12:18:46.847716 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.497s	user 0.016s	sys 0.023s
I20260709 12:18:46.850835 28551 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.852152 28419 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.851907 28769 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:46.720733 25637 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.456257 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.863927 25705 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:46.865448 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:46.722075 25778 raft_consensus.cc:2368] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
W20260709 12:18:46.868274 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1649) took 1900 ms (client timeout 2206 ms). Trace:
I20260709 12:18:46.366902 25779 rpcz_store.cc:276] 0709 12:18:44.922319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.923898 (+  1579us) service_pool.cc:224] Handling call
0709 12:18:44.924136 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.036463 (+112327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.302004 (+1265541us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54001a86d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:46.302302 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256694656}
I20260709 12:18:46.723568 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2249) took 1394 ms. Trace:
I20260709 12:18:46.524814 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:46.871440 25784 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.869946 25711 rpcz_store.cc:276] 0709 12:18:45.328799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.332593 (+  3794us) service_pool.cc:224] Handling call
0709 12:18:45.333408 (+   815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:45.333779 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:46.247439 (+913660us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b1800415900. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:46.368018 (+120579us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.547278 (+179260us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.547763 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.547889 (+   126us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:46.547933 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.547999 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.722760 (+174761us) raft_consensus.cc:1685] finished
0709 12:18:46.722964 (+   204us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.723310 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2356,"prepare.queue_time_us":237151,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":195645,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":14737,"spinlock_wait_cycles":16544128,"thread_start_us":1151010,"threads_started":2,"wal-append.queue_time_us":228679}
I20260709 12:18:46.523403 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:46.535387 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:46.873586 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:46.874912 28517 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:46.875531 25637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2028) took 2346 ms (client timeout 2119 ms). Trace:
I20260709 12:18:46.876276 25638 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:46.876010 28517 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.484333 25706 raft_consensus.cc:2410] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:18:46.879343 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2250) took 1550 ms (client timeout 1752 ms). Trace:
W20260709 12:18:46.879839 25706 rpcz_store.cc:269] 0709 12:18:45.329180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.329297 (+   117us) service_pool.cc:224] Handling call
0709 12:18:46.879134 (+1549837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.480386 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:46.545207 25750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.023s	sys 0.022s
W20260709 12:18:46.886425 25750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.023s	sys 0.023s
I20260709 12:18:46.886824 25708 raft_consensus.cc:2410] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:46.851967 28551 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.888070 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:46.889537 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:46.549849 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 120048us
I20260709 12:18:46.896742 28089 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:46.897755 28089 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.898344 28698 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:46.484301 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1651) took 1509 ms. Trace:
I20260709 12:18:46.483179 26335 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:46.515129 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 551) took 1950 ms. Trace:
W20260709 12:18:46.586229 27534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.044s	sys 0.088s
I20260709 12:18:46.567420 28346 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:46.640157 28587 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:46.909228 26338 rpcz_store.cc:276] 0709 12:18:44.564767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.564920 (+   153us) service_pool.cc:224] Handling call
0709 12:18:44.586651 (+ 21731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.364663 (+1778012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.514851 (+150188us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400471654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:46.515063 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2368270080}
I20260709 12:18:46.910211 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.590370 28623 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.911628 28857 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:46.912560 28857 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.913682 28623 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.648253 28726 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.591724 28734 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:46.910310 28587 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:46.899008 28698 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:46.916416 28734 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.659566 28736 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:46.741276 25752 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.003s	sys 0.006s
I20260709 12:18:46.720775 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2031) took 2049 ms. Trace:
I20260709 12:18:46.745688 25783 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:46.749209 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2270) took 1384 ms. Trace:
I20260709 12:18:46.592049 28730 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:46.919240 25635 rpcz_store.cc:276] 0709 12:18:44.670743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.802681 (+131938us) service_pool.cc:224] Handling call
0709 12:18:44.803523 (+   842us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.814645 (+ 11122us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:44.814689 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:44.815276 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:18:45.120625 (+305349us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:45.120679 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.120772 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.719596 (+1598824us) raft_consensus.cc:1685] finished
0709 12:18:46.719714 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.720421 (+   707us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10655,"raft.queue_time_us":12183,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5202,"thread_start_us":277279,"threads_started":1,"wal-append.queue_time_us":420180}
I20260709 12:18:46.923894 28730 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:46.473479 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.023s	sys 0.042s
I20260709 12:18:46.764448 28574 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:46.931999 25752 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.003s	sys 0.006s
I20260709 12:18:46.767454 28598 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:46.768906 26339 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:46.900734 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:46.784874 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:46.722337 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 416) took 1876 ms. Trace:
I20260709 12:18:46.900933 28089 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.596479 28596 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.644357 28757 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.939308 28517 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.939639 28596 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.940514 25713 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:46.941610 28757 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:46.609093 28783 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.706s	user 0.008s	sys 0.006s
I20260709 12:18:46.643179 28780 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:46.943003 28780 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:46.944860 28780 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.663125 28789 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:46.618824 28806 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.722s	user 0.008s	sys 0.007s
I20260709 12:18:46.947557 28806 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:46.950075 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 801861us
I20260709 12:18:46.585609 28814 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.583400 28818 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:46.652710 28821 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.445s	user 0.011s	sys 0.000s
I20260709 12:18:46.644089 28825 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.624212 28846 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.585275 28847 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:46.655607 28850 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.097s	user 0.002s	sys 0.000s
I20260709 12:18:46.655035 28869 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.413064 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:46.438045 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.516106 25787 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:46.907063 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 55221.4us
W20260709 12:18:46.868862 25778 rpcz_store.cc:269] 0709 12:18:44.974459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.990411 (+ 15952us) service_pool.cc:224] Handling call
0709 12:18:46.867846 (+1877435us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54001a81d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:46.868115 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3978429184}
W20260709 12:18:46.876061 25637 rpcz_store.cc:269] 0709 12:18:44.529147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.598116 (+ 68969us) service_pool.cc:224] Handling call
0709 12:18:46.875160 (+2277044us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b54003007d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:46.875433 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4836089088}
W20260709 12:18:46.886894 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.951318 25708 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:46.901841 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:46.896096 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1691) took 1339 ms. Trace:
I20260709 12:18:46.900158 28805 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:46.890684 28559 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:46.906914 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.896651 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:46.899080 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.895792 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.963810 25633 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:46.917682 28587 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.919718 25710 rpcz_store.cc:276] 0709 12:18:45.364159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.414427 (+ 50268us) service_pool.cc:224] Handling call
0709 12:18:46.748643 (+1334216us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400105c54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:46.748910 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2335937408}
I20260709 12:18:46.970158 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.915573 25782 rpcz_store.cc:276] 0709 12:18:44.975220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.055894 (+ 80674us) service_pool.cc:224] Handling call
0709 12:18:45.094559 (+ 38665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.362525 (+1267966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.484013 (+121488us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b540011f154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:46.484230 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221291392}
I20260709 12:18:46.971258 25708 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:46.780798 28725 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:18:46.796591 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.943048 28783 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.972591 28886 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.973354 28886 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.974990 28790 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.975214 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.975982 25634 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:46.976326 25637 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.976418 28791 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:46.977042 28878 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
I20260709 12:18:46.977339 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:46.977850 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.978169 25781 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:46.935544 28787 raft_consensus.cc:476] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Not starting pre-election -- already a leader
W20260709 12:18:46.641940 28811 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.529s	user 0.013s	sys 0.000s
I20260709 12:18:46.971057 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 716) took 1998 ms. Trace:
I20260709 12:18:46.971132 25632 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:46.952685 28789 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.978466 25634 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.998020 25707 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:46.888082 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.951481 28769 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.999307 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.977388 28791 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.000792 25633 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.972086 28725 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:47.003422 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:47.003563 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2123) took 2481 ms (client timeout 2231 ms). Trace:
I20260709 12:18:47.004323 26341 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:47.004220 28725 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.005537 28725 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:46.966526 28734 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:46.933434 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 1.5052e+06us
I20260709 12:18:47.007299 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:46.968778 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:46.934543 25709 rpcz_store.cc:276] 0709 12:18:44.845619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.848316 (+  2697us) service_pool.cc:224] Handling call
0709 12:18:44.848768 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.849244 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.849286 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:44.849865 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 12:18:45.121549 (+271684us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:45.121598 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.122095 (+   497us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.721514 (+1599419us) raft_consensus.cc:1685] finished
0709 12:18:46.721572 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.721927 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2859,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":123,"thread_start_us":271326,"threads_started":1,"wal-append.queue_time_us":442791}
I20260709 12:18:47.000304 28769 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.965755 28732 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:47.002550 28726 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:47.003111 28791 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:46.998958 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:46.958000 25630 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.957463 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:46.982370 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82049.8us
I20260709 12:18:46.979423 25629 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.918320 28698 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:46.971374 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.986449 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:46.990378 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:46.987303 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:46.987737 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 486) took 1612 ms. Trace:
W20260709 12:18:47.209096 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.297161 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 393964us
I20260709 12:18:46.997638 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 710) took 2112 ms. Trace:
I20260709 12:18:47.345026 25713 raft_consensus.cc:2410] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:47.279075 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:46.992398 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.977378 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.309719 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:46.994951 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:46.997691 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:46.988945 25752 leader_election.cc:336] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.950s (SENT)
I20260709 12:18:46.958920 25638 rpcz_store.cc:276] 0709 12:18:45.556779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.557507 (+   728us) service_pool.cc:224] Handling call
0709 12:18:46.875890 (+1318383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.876750 (+   860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.895879 (+ 19129us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003007d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:46.896020 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3031065984}
W20260709 12:18:47.196134 25707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2210) took 1554 ms (client timeout 1851 ms). Trace:
W20260709 12:18:47.004170 25781 rpcz_store.cc:269] 0709 12:18:44.522207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.990121 (+467914us) service_pool.cc:224] Handling call
0709 12:18:47.003252 (+2013131us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54000eea54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:47.003459 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4448046976}
I20260709 12:18:46.997731 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2211) took 1322 ms. Trace:
I20260709 12:18:46.981343 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:47.396634 25707 rpcz_store.cc:269] 0709 12:18:45.641816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.029588 (+387772us) service_pool.cc:224] Handling call
0709 12:18:47.116221 (+1086633us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:47.286367 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:47.404249 25637 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:47.292906 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.363438 25787 rpcz_store.cc:276] 0709 12:18:45.375562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.375773 (+   211us) service_pool.cc:224] Handling call
0709 12:18:45.376171 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.965772 (+1589601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.987182 (+ 21410us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000870d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:46.987496 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2621064064}
I20260709 12:18:47.303194 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:47.274350 28748 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.972029 28857 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.409852 26343 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:47.297874 28649 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:47.432577 26341 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:46.992190 28886 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.991358 28587 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:46.991333 28783 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:46.989677 28551 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.410259 26342 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:47.274966 28780 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:46.986882 28772 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.990686 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:47.410293 25708 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:47.366016 26339 rpcz_store.cc:276] 0709 12:18:44.885234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.885971 (+   737us) service_pool.cc:224] Handling call
0709 12:18:44.886289 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.939913 (+2053624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.997160 (+ 57247us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b54006caf54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:46.997520 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4263664384}
I20260709 12:18:47.286619 28596 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:46.986763 28757 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:47.426425 25635 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:47.430991 26344 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:47.435624 26340 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:47.436120 25634 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
W20260709 12:18:47.439216 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.979626 28773 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.432159 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:46.980250 28570 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.979941 28346 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:47.286141 28787 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:46.993886 28814 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:47.408736 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:46.991963 28089 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.446844 25632 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:47.460448 25712 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:47.461849 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:47.463887 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.474231 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:47.486227 25603 connection.cc:256] RPC call timeout handler was delayed by -1.3964674569999715s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
W20260709 12:18:47.509336 25603 connection.cc:256] RPC call timeout handler was delayed by -1.3758714759999293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 12:18:47.442915 28885 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:18:47.487108 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.447777 28698 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:47.474905 26337 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:46.987717 28806 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.442438 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:47.473853 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:47.470525 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:47.463245 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:47.442822 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:47.302368 28805 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.480887 28757 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.518000 26341 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:46.995381 28789 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.404640 25704 rpcz_store.cc:276] 0709 12:18:45.672134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.487513 (+815379us) service_pool.cc:224] Handling call
0709 12:18:46.487804 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.488002 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:46.519373 (+ 31371us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.969097 (+449724us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b1800064f00. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:46.938050 (+-31047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.994010 (+ 55960us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:46.994065 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.994152 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.994314 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59367,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":1539,"raft.queue_time_us":7764,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":181,"spinlock_wait_cycles":18055552,"thread_start_us":339946,"threads_started":1,"wal-append.queue_time_us":365594}
I20260709 12:18:47.285889 28845 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:47.463284 28846 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:47.468511 28574 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.535s	user 0.005s	sys 0.005s
I20260709 12:18:47.466740 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:47.457706 28519 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 2.068s	user 0.011s	sys 0.015s
I20260709 12:18:47.475843 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.474462 28847 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.459141 28769 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:47.529740 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55599.9us
I20260709 12:18:46.979902 28559 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.517304 28825 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.442658 28734 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:47.535151 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.520557 26339 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:46.982972 26335 rpcz_store.cc:276] 0709 12:18:44.972690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.981115 (+  8425us) service_pool.cc:224] Handling call
0709 12:18:45.117391 (+136276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.909316 (+1791925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.936150 (+ 26834us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540044b454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:46.970786 (+ 34636us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3047529216}
W20260709 12:18:47.523219 25603 connection.cc:256] RPC call timeout handler was delayed by -1.3461714759998813s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
I20260709 12:18:47.444075 28732 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.461689 28649 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:47.524504 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:47.533826 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 112699us
I20260709 12:18:47.444603 28726 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.535888 25787 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:18:47.455582 28748 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.521847 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:46.981465 28811 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.536660 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:47.522733 28757 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:46.975976 28790 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.539537 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1653) took 1006 ms. Trace:
I20260709 12:18:47.540933 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.525463 28857 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.565045 28811 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.565402 28857 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.566115 28811 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:47.541723 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:47.585373 26341 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:47.585558 26342 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:47.471750 28772 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:47.546456 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.548579 25603 connection.cc:256] RPC call timeout handler was delayed by -1.2837999400001081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:18:47.588434 25603 connection.cc:256] RPC call timeout handler was delayed by -1.0592492020000464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 12:18:47.552968 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:47.540019 28847 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.536751 28519 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.555763 28783 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:47.535583 28734 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.558319 26337 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:47.606415 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:47.558961 25713 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:47.538508 28886 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.547451 28346 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.535508 28787 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.532583 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:47.566988 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 124868us
I20260709 12:18:47.541029 28570 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.613031 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.570420 26339 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:47.571009 28726 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.452893 28858 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
W20260709 12:18:47.568802 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.596535 28551 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:47.575495 25781 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:18:47.541241 28825 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.580466 28559 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.624686 26340 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:47.568400 28805 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.593041 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:47.586956 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 139852us
I20260709 12:18:47.565164 28821 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:47.596436 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 607517us
I20260709 12:18:47.597963 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.568140 28857 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.556074 28649 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.535799 28773 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.550793 25780 rpcz_store.cc:276] 0709 12:18:46.520447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.522860 (+  2413us) service_pool.cc:224] Handling call
0709 12:18:46.553171 (+ 30311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.553417 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:46.896477 (+343060us) log.cc:844] Serialized 14 byte log entry
0709 12:18:47.401861 (+505384us) log.cc:844] Serialized 14 byte log entry
0709 12:18:47.421391 (+ 19530us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:47.428409 (+  7018us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:47.476485 (+ 48076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:47.502872 (+ 26387us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.526920 (+ 24048us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39385,"prepare.queue_time_us":731397,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3790,"raft.queue_time_us":83974,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":26808,"spinlock_wait_cycles":272964096,"thread_start_us":436635,"threads_started":2,"wal-append.queue_time_us":609224}
I20260709 12:18:47.615671 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.617612 25778 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:47.551168 28089 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.616961 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:47.621105 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.620739 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.617993 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.574297 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:47.623119 28793 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.903s	user 0.000s	sys 0.016s
I20260709 12:18:47.623020 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:47.533409 28791 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:47.621723 25704 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:47.654775 25705 raft_consensus.cc:2410] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:47.625900 28551 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.695067 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:47.635439 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 644901us
W20260709 12:18:47.637946 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45070.4us
I20260709 12:18:47.648701 25713 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:47.702723 25632 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:47.674031 28734 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.638897 28559 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:18:47.657698 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46266.8us
I20260709 12:18:47.708220 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.710337 25603 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:47.658854 28888 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:18:47.665463 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:47.668324 28732 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.613044 28886 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.705928 28551 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:47.726413 28791 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:47.594074 28846 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:47.582460 28845 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.683410 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:47.687614 28519 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.687968 28787 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.688205 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:47.690058 28825 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.690763 28772 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:47.691053 28570 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.692138 28789 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.600433 28783 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:47.757467 28783 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:47.592638 28811 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.703321 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.704983 25631 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:47.721844 25713 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:47.730583 26341 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:47.735455 26337 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:47.736378 25712 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:47.741034 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:47.744577 25708 raft_consensus.cc:2410] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:47.734627 25706 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:47.693342 28748 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.696312 28726 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:47.712270 28857 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:47.728886 28559 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:47.760744 25638 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:47.776481 25707 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:47.763953 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2220) took 1014 ms. Trace:
I20260709 12:18:47.767614 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.779477 25637 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:47.787720 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:47.796945 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 86753.6us
I20260709 12:18:47.775000 28805 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:47.813540 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 361265us
I20260709 12:18:47.775568 28857 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:47.822259 28346 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:47.823679 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:47.823846 28814 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.824076 25636 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:47.824514 28814 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.796025 25712 rpcz_store.cc:276] 0709 12:18:46.749255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.935124 (+185869us) service_pool.cc:224] Handling call
0709 12:18:47.763716 (+828592us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:47.839246 25676 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:47.841096 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 194034us
I20260709 12:18:47.848758 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:47.849946 25704 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:47.853328 25707 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:47.794735 28821 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:47.830662 28519 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:47.831144 28857 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:47.869153 25634 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:47.869462 26340 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:47.870662 26342 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: [3.6-6.7]
I20260709 12:18:47.877079 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.810467 28789 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 12:18:47.882949 28889 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.340s	user 0.004s	sys 0.001s
W20260709 12:18:47.898825 28890 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.250s	user 0.001s	sys 0.004s
I20260709 12:18:47.901352 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:47.907447 28773 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:47.912492 25677 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:47.914062 28896 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:47.915066 28896 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:47.901049 26344 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:47.884814 28559 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.885738 28821 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.890748 28811 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.892632 28845 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.882920 28089 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:47.882895 28846 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.917254 25704 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:47.926541 26339 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:18:47.930716 28894 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:47.931386 28894 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:47.932644 28894 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.915584 28896 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.915772 28787 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:47.849614 28805 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:47.881062 28790 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:47.956254 26338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.092s	sys 0.119s
W20260709 12:18:47.956743 26338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.093s	sys 0.119s
I20260709 12:18:47.934388 28894 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:47.965310 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:47.966729 26343 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:47.906337 28551 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:47.968580 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.971324 28847 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.971784 28551 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:47.973242 25638 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:47.958946 28846 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:47.976956 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.977119 28846 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:47.977540 28846 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.980899 28886 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.982035 28805 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:47.957379 28811 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:18:47.965994 25635 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:47.953413 28896 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:47.990665 28787 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:47.991110 28845 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:47.992190 28845 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:47.992569 28845 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:47.991815 28787 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:48.014456 25637 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:48.013424 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1700) took 1056 ms. Trace:
I20260709 12:18:47.966818 28814 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:48.030059 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 218137us
I20260709 12:18:48.017509 25636 rpcz_store.cc:276] 0709 12:18:46.949470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.008133 (+ 58663us) service_pool.cc:224] Handling call
0709 12:18:47.275836 (+267703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.824462 (+548626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.005750 (+181288us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54001c66d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:48.006024 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259868032}
W20260709 12:18:48.013144 28725 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.007s	user 0.013s	sys 0.004s
I20260709 12:18:48.006986 28825 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:48.009233 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 789) took 1003 ms. Trace:
I20260709 12:18:48.031322 26335 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:48.030917 28780 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:48.029841 28789 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:48.017815 28858 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:48.056919 28790 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:48.057440 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.096289 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 56474.1us
W20260709 12:18:48.046586 28587 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.003s
I20260709 12:18:48.102383 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.103983 28773 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.104992 28825 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:48.104724 28773 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:48.105451 28825 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.105880 28825 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.105932 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.106685 28596 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:48.106992 28748 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:48.107502 28748 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.107122 28790 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:48.108412 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.107899 28748 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:48.110086 28790 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:48.099692 28858 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:48.106678 28789 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.030313 28783 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:48.103122 28698 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.576s	user 0.003s	sys 0.010s
I20260709 12:18:48.126348 28596 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:48.146147 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.147141 25636 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:48.149210 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.150153 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.150857 25638 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:48.152400 25631 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:18:48.147101 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 500731us
I20260709 12:18:48.133363 28596 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:48.049691 26338 rpcz_store.cc:276] 0709 12:18:47.006033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.006512 (+   479us) service_pool.cc:224] Handling call
0709 12:18:47.006895 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.007328 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:47.261193 (+253865us) log.cc:844] Serialized 14 byte log entry
0709 12:18:47.986818 (+725625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:47.986901 (+    83us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:48.003463 (+ 16562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.003662 (+   199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.009144 (+  5482us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29523,"prepare.queue_time_us":444469,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":64,"raft.queue_time_us":32426,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":16047,"thread_start_us":897305,"threads_started":2}
W20260709 12:18:48.156057 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 332487us
I20260709 12:18:48.147212 25634 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:48.159850 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.051757 28845 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:48.162050 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.122068 28780 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.165925 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.173288 25636 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:48.174599 28806 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:48.175115 28806 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.175530 28806 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:48.159672 28847 raft_consensus.cc:2764] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:18:48.180974 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.183182 25630 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:48.184151 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.184993 25633 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:48.168390 28519 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.181131 28734 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:18:48.186959 28519 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.188982 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.189764 25631 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:48.190533 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.192219 28806 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.193014 25634 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:48.195194 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.195894 25634 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:48.200266 28806 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:48.201220 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.203318 25632 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:48.201503 28858 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.204655 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.205132 28858 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.210405 25634 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:48.213944 28773 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:48.214229 25638 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:48.215205 28845 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:48.225540 28783 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:48.227496 28783 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.228035 28783 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:48.229736 28757 consensus_meta.cc:287] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.001s
I20260709 12:18:48.230232 28757 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.230722 28783 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:48.238830 28596 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.239851 28757 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:48.240113 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:48.229681 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.240792 25638 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:48.247849 28780 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:48.252616 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 150671us
I20260709 12:18:48.252964 28780 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:48.253578 28780 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:48.254837 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:48.255844 28780 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:48.255810 28587 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:48.256862 28089 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:48.257215 28587 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:48.257773 28587 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:48.258249 28821 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:48.259214 25638 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:48.259459 28587 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.261234 28587 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:48.256295 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:48.264833 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:48.265795 28780 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:48.266650 28780 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:48.168372 28698 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:48.267416 28780 consensus_queue.cc:1059] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:48.267324 28698 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.268512 28780 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:48.268869 28780 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:48.269883 28780 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.271136 28780 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.271817 25676 leader_election.cc:400] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:48.272364 25676 leader_election.cc:403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:18:48.275660 28748 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.279794 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.276769 28089 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:48.280431 25631 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:18:48.282066 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:48.261372 28789 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:48.299002 28726 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:48.298703 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.300055 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.300462 25635 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:48.301677 28089 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:18:48.302700 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 140982us
W20260709 12:18:48.340049 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 429697us
I20260709 12:18:48.339159 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:48.341938 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 447423us
I20260709 12:18:48.345777 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.338872 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:48.353704 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 73461.9us
I20260709 12:18:48.351795 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.338127 28748 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:48.359764 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.361454 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:48.376642 28805 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:48.377213 28805 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:48.353368 28757 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:48.377087 28896 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.377967 28757 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:48.378322 28805 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.379150 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:48.381538 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 61180us
W20260709 12:18:48.383311 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.356750 28806 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:48.391703 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:48.392275 26338 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:48.341521 28789 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:18:48.396395 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:48.398411 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.373927 28734 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:48.391801 28519 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:48.394958 28847 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:48.378479 28757 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:48.410562 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:48.411224 28559 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:48.378870 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.413936 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:48.379842 28896 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:48.426888 25631 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:48.428798 26336 raft_consensus.cc:2376] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 7. Re-sending same reply.
I20260709 12:18:48.440057 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:48.431273 28734 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:48.417352 28089 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:48.419766 28773 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:48.422158 28806 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:48.421929 28847 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.442126 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:48.453495 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:48.453804 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.467442 25782 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:48.455852 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.449102 28519 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:48.457238 28734 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.447248 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:48.446102 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:48.469334 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.418068 28846 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:48.466689 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.454501 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:48.481834 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 135769us
I20260709 12:18:48.470009 25787 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:48.487921 25780 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:48.500288 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.512152 28649 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:48.513070 28649 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.516613 28649 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:48.492110 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:48.414450 28825 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.449357 28089 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:48.432356 28857 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:48.458884 28773 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:48.487699 26335 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:18:48.544140 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 130296us
I20260709 12:18:48.471292 28847 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:48.536583 25632 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:48.554797 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.543453 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.558115 25637 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:48.469337 28519 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:48.471915 28900 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:48.552634 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.552958 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.561064 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.561625 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.562201 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.553622 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.524115 28790 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:48.564581 26338 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:48.564960 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.527521 28789 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:48.566157 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 112768us
I20260709 12:18:48.526327 28698 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:48.543553 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.578584 25783 raft_consensus.cc:2410] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:48.571957 28857 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:48.574715 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 127573us
I20260709 12:18:48.544862 28773 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.466871 28896 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:18:48.592566 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.565522 28519 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:48.594717 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.556533 28559 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:48.576207 28789 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:48.603951 25781 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:48.606906 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.603680 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2224) took 1302 ms. Trace:
I20260709 12:18:48.628165 26337 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:18:48.608574 28910 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:18:48.615474 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.616381 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.639845 25787 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:48.617291 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:48.627388 25711 rpcz_store.cc:276] 0709 12:18:47.306035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.307468 (+  1433us) service_pool.cc:224] Handling call
0709 12:18:47.307917 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:47.653737 (+345820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:47.653785 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:47.755756 (+101971us) log.cc:844] Serialized 26 byte log entry
0709 12:18:48.281860 (+526104us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:48.452562 (+170702us) log.cc:844] Serialized 14 byte log entry
0709 12:18:48.460606 (+  8044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.596118 (+135512us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400104854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:48.596174 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:48.596243 (+    69us) raft_consensus.cc:1685] finished
0709 12:18:48.603367 (+  7124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.603595 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":126556,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":106,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":90,"spinlock_wait_cycles":763843328,"thread_start_us":587099,"threads_started":2,"wal-append.queue_time_us":635381}
I20260709 12:18:48.640969 25778 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:48.616436 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:48.619225 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.621142 28825 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:48.621560 28858 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:48.648692 25782 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:48.645653 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:48.593226 28900 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:48.671238 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:48.680095 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 75577.5us
I20260709 12:18:48.638847 28789 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:48.678323 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.683919 25784 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:48.600781 28845 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:18:48.677507 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.685356 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.695127 25787 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:48.690944 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:48.692114 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55452.8us
I20260709 12:18:48.696171 25783 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:48.694417 25786 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:48.690846 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.709966 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:48.711050 25785 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.710908 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.711768 25783 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:48.716126 28900 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.719298 28894 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:48.721751 28894 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:48.723661 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.701994 28698 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:48.730502 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:48.599973 28806 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:48.746886 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 152532us
I20260709 12:18:48.750854 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.751683 25636 raft_consensus.cc:2368] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:48.751752 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.752471 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.752748 25635 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:48.753378 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.753774 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.754558 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.755620 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.755771 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:48.756716 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.757464 25634 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:48.757977 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:48.758177 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 131110us
I20260709 12:18:48.759353 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:48.760874 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:48.762346 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:48.763495 28858 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:48.764843 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.766594 25638 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:48.758215 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.767287 25634 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 3.
I20260709 12:18:48.772554 25636 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:48.771142 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.776942 25632 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.778420 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.779570 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.789311 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.791543 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:48.793273 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:48.797348 25603 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:48.801213 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:48.802086 25629 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:18:48.802476 25629 pending_rounds.cc:85] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 5
I20260709 12:18:48.803237 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:48.803989 25637 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:48.806668 26341 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:48.722342 28894 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.726509 28725 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:18:48.807080 25633 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:48.763942 28858 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:48.772544 28857 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:48.802707 25629 pending_rounds.cc:107] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:18:48.806939 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.822310 25634 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:48.821624 25629 raft_consensus.cc:2889] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:48.775491 28846 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:48.714413 28089 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:48.826508 28089 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:48.827358 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.826828 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:48.829983 25752 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:48.801446 28845 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:48.800881 28698 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:48.864445 28787 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:48.821655 28858 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:48.865360 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2281) took 1557 ms. Trace:
I20260709 12:18:48.874155 25638 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:48.765395 28559 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:48.882995 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 58757us
I20260709 12:18:48.862097 28922 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.861755 28915 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.865103 28846 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:48.854754 28725 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:48.862159 28845 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:48.888835 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 94366.6us
I20260709 12:18:48.862010 28903 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
W20260709 12:18:48.895036 28924 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:18:48.867054 28930 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:48.879694 28698 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:48.900638 26339 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
I20260709 12:18:48.906572 26339 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
W20260709 12:18:48.908828 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 112556us
W20260709 12:18:48.910399 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 113113us
I20260709 12:18:48.911841 25603 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:48.904433 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:48.864485 28783 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:18:48.913679 25785 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:48.914119 25632 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:48.919376 25778 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:18:48.915640 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:48.930160 28773 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:48.932884 25779 raft_consensus.cc:2410] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:48.906200 28930 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:48.939162 28930 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:48.946370 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.901811 28725 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:48.947803 28725 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:48.914384 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:48.907636 28857 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:48.987596 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 73304.9us
I20260709 12:18:48.913537 28698 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:49.022912 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 107353us
I20260709 12:18:49.023794 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:48.988940 28940 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:48.995983 28911 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.291s	user 0.000s	sys 0.002s
I20260709 12:18:49.028926 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:48.896638 28922 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.034185 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:49.034935 28940 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:48.906985 28915 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:49.053007 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 141273us
W20260709 12:18:49.065960 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46844.1us
I20260709 12:18:49.066380 25603 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:49.076534 28757 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.001s
I20260709 12:18:49.077060 28757 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.078475 26340 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:49.057307 28941 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:49.049881 28938 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:49.058786 28934 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:49.023957 28845 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.093245 25603 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:49.062608 28894 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.068809 28944 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:49.119781 28940 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:49.121364 28940 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.094795 28934 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.090991 28938 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:49.128253 28783 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:18:49.141709 28757 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:49.097121 28698 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:49.143543 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.126219 28941 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:49.115150 28944 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:49.146322 28847 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.559s	user 0.004s	sys 0.008s
W20260709 12:18:49.157006 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.157346 28954 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:49.159868 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 315046us
W20260709 12:18:49.163636 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 95095.2us
I20260709 12:18:49.139132 28934 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:49.138692 28938 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:49.167562 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 86919.8us
I20260709 12:18:49.160218 28941 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:49.180115 28932 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 12:18:49.181965 28935 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:18:49.183013 28945 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:18:49.184309 28929 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 12:18:49.184736 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:49.188186 28920 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:49.189917 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:49.191772 28920 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:49.192178 28920 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.192539 28920 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.168684 28903 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:49.198480 28903 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.166718 28847 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.201545 28847 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.202441 25635 raft_consensus.cc:2368] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6.
I20260709 12:18:49.198925 28903 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.161290 28944 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:49.212371 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 178255us
I20260709 12:18:49.102342 28857 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.214395 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:49.167071 28845 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:49.219617 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:49.220052 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 126952us
I20260709 12:18:49.222005 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:49.223416 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:49.223371 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45311.6us
W20260709 12:18:49.218775 28519 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.614s	user 0.003s	sys 0.008s
I20260709 12:18:49.226922 28950 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:49.229509 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.206949 28941 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.232228 25780 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:49.219380 28846 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:49.234344 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.235062 25785 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:18:48.874514 25709 rpcz_store.cc:276] 0709 12:18:47.307779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.714478 (+406699us) service_pool.cc:224] Handling call
0709 12:18:47.778902 (+ 64424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.872729 (+ 93827us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:48.314160 (+441431us) log.cc:844] Serialized 14 byte log entry
0709 12:18:48.863380 (+549220us) log.cc:844] Serialized 14 byte log entry
0709 12:18:48.863737 (+   357us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:48.863853 (+   116us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:48.864130 (+   277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.865022 (+   892us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.865287 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8339,"prepare.queue_time_us":488699,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":83994,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":11409,"thread_start_us":816967,"threads_started":3,"wal-append.queue_time_us":485844}
I20260709 12:18:49.231551 28950 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:49.241428 28950 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:49.232955 28519 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.245551 28519 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.245791 25785 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
W20260709 12:18:49.247573 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.247383 28757 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:49.247814 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:49.249259 28757 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:49.249776 28757 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:49.249987 28780 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:49.250479 28780 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:49.250864 28757 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.251389 28780 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.252879 28780 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:49.262763 25780 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:18:49.264096 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:49.219147 28725 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.233381 28954 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:49.254634 28698 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:49.223708 28930 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.279253 25630 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:18:49.280406 28789 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 12:18:49.282869 28789 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.285264 28789 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.286536 28954 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:49.287572 28954 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.292048 28930 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.293167 28858 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.294003 28930 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:49.295797 28725 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.304522 28922 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:49.306448 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.319612 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.320001 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.307824 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 96117.4us
I20260709 12:18:49.308405 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.321666 25704 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.322782 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.323541 25704 raft_consensus.cc:2410] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:49.323840 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:49.324421 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.325232 25704 raft_consensus.cc:2410] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:49.309120 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.309454 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.310145 28931 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:49.310415 28926 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.313884 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.310593 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.310925 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.355803 25707 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 3.
I20260709 12:18:49.314886 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.308079 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:49.311481 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.310762 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.311379 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.357376 25709 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:18:49.357481 25705 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.310925 28894 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.316938 28903 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:49.332978 28921 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.472s	user 0.000s	sys 0.004s
W20260709 12:18:49.327077 28916 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:18:49.335312 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.368832 25704 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.369776 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.334113 25713 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:49.338011 25632 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:49.356741 25706 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.318377 28858 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.376722 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.377403 25704 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
W20260709 12:18:49.378225 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.357584 28698 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:49.380869 25633 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:18:49.381623 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.382637 25713 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:49.382776 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.383518 25707 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:49.384078 25635 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:49.385001 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:49.386276 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:49.396243 25630 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:49.395424 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:49.397511 25633 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:49.397977 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.400542 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.401242 25785 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:49.372267 28896 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.404107 25712 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:49.407226 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.408046 25786 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:49.405455 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:49.412532 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.415328 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:49.366339 28894 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.418781 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:49.423437 25630 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:49.422290 25783 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:49.364738 28926 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.421168 28725 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:49.427193 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.427109 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:49.432736 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 210838us
I20260709 12:18:49.434185 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:49.434720 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.436142 25712 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.439184 25784 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:18:49.439653 25784 pending_rounds.cc:85] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 2
I20260709 12:18:49.439424 28920 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.440027 25784 pending_rounds.cc:107] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:18:49.440455 25784 raft_consensus.cc:2889] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:49.441833 28920 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:49.445997 28947 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:49.446959 28947 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:49.426918 28789 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:18:49.447912 28953 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:49.448266 28789 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:49.448774 28789 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:49.449287 28949 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:18:49.429868 28587 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:18:49.369907 28931 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.471428 28089 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:49.476972 25713 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:49.448871 28941 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.416942 28649 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:49.486505 25785 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:49.493383 25704 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.426642 28927 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:49.501464 28089 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:49.504454 28089 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:49.511305 28089 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.514914 25634 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:18:49.508270 28964 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:18:49.520213 28938 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.522145 25629 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:49.509641 28927 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.448725 28913 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.147s
I20260709 12:18:49.500435 28698 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.451790 28896 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:49.537447 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.537976 28089 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:49.564411 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:49.564705 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:49.565171 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.567011 25708 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:49.580008 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.580801 25708 raft_consensus.cc:2410] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 12:18:49.538563 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 104470us
I20260709 12:18:49.582808 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:49.530462 25630 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:18:49.585011 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.585271 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:49.587391 25631 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:49.546905 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.587108 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:49.560268 25706 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:49.588186 25712 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:49.574218 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 293272us
I20260709 12:18:49.567756 25786 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:49.570539 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.590737 25779 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.591153 25783 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:49.591854 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:49.593818 25637 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:49.569512 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.595270 25635 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 5: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:49.599177 25677 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:49.602941 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:49.606947 25603 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:49.543040 28913 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:49.538825 28903 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.611553 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.619479 25708 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:49.543141 28941 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.543599 28938 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.545301 28931 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.552583 28927 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.551131 28894 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.553761 28926 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.542771 28649 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:49.553701 28896 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.543018 28930 raft_consensus.cc:2764] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:49.634392 25711 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:49.635136 28954 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:49.635689 25710 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:18:49.647361 25710 pending_rounds.cc:85] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 6
I20260709 12:18:49.551311 28950 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.638685 28931 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:49.640128 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.543815 28965 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.122s
I20260709 12:18:49.609632 28845 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:49.604894 28955 raft_consensus.cc:2764] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:18:49.608953 28922 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:49.623374 28913 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:49.647640 25710 pending_rounds.cc:107] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:18:49.664496 25710 raft_consensus.cc:2889] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:49.638552 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 568) took 1145 ms. Trace:
I20260709 12:18:49.546203 28934 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.660913 25632 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:49.643177 28806 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.666534 25712 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:49.679689 25787 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:49.666137 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 305340us
I20260709 12:18:49.677858 28787 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:49.665091 26340 rpcz_store.cc:276] 0709 12:18:48.493502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.493744 (+   242us) service_pool.cc:224] Handling call
0709 12:18:48.494087 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.079069 (+584982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.638223 (+559154us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b54005a7154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:49.638462 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1343343488}
I20260709 12:18:49.686702 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 570) took 1180 ms. Trace:
I20260709 12:18:49.687098 26336 rpcz_store.cc:276] 0709 12:18:48.499415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.576866 (+ 77451us) service_pool.cc:224] Handling call
0709 12:18:49.679399 (+1102533us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54004812d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:49.679531 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195232512}
I20260709 12:18:49.690897 28974 raft_consensus.cc:2764] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:49.692616 28974 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:49.693114 28974 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:49.694813 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.697108 28913 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.698334 28773 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.695017 28787 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:49.664625 28922 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:49.707862 25783 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
W20260709 12:18:49.705500 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 102651us
I20260709 12:18:49.708786 25752 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:49.664526 28845 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:49.706708 25779 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:49.663007 28965 consensus_queue.cc:1243] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:18:49.693953 28974 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.702195 28922 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:49.704058 28934 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.661970 28931 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.698457 28913 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:49.660813 28944 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.721727 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.737365 25712 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:49.735297 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.739495 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.740078 25712 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.741044 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.741645 25710 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:49.733054 28980 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.743076 28980 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.713351 28649 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.744951 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.745573 25708 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.747960 28789 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.749409 28789 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:49.748462 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:49.750870 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.751456 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.751801 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.754878 25706 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:49.755465 25712 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:49.756554 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.759712 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.766572 28968 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:49.750939 25711 raft_consensus.cc:2376] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 7. Re-sending same reply.
I20260709 12:18:49.767642 28898 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:49.768133 28898 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:49.768496 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.769151 25711 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:49.769068 28898 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.770264 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.698562 28806 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.768991 28845 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.775663 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.787720 25707 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
W20260709 12:18:49.778659 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.789261 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.774618 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.778719 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.792115 25705 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.781693 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.794195 25712 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:49.797436 25786 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:49.799448 25704 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:18:49.799983 25603 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:49.800678 25708 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:49.802170 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.802954 25708 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:49.805624 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.806586 25783 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:49.807742 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.779078 28968 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:49.811367 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:49.780570 28806 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:49.818145 28954 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:49.818792 28954 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:49.818418 28968 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.819247 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.819765 28954 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:49.820616 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.821333 25778 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.822857 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.823771 25782 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:49.824000 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.825059 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.825227 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.826614 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.826628 25781 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:49.828677 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.828752 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.829367 25782 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:49.829833 25781 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:49.830753 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.831291 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:49.831863 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.832325 28825 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:49.832710 28825 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:49.832887 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.833105 28825 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:49.833575 25781 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:49.827440 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.834336 25786 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:49.822176 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.839025 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.830868 26337 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:49.840485 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:49.770457 28898 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:49.835316 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.860544 25786 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:49.844391 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:49.861320 26341 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:49.835027 25778 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:49.861824 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.781836 28950 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:49.852957 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.834446 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.870597 25707 raft_consensus.cc:2410] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:49.871824 25710 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:49.775224 28980 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.866991 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:49.772156 28915 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.864089 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.880021 25782 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:49.883949 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.891690 25783 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:49.895416 25704 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:49.895747 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.896795 25603 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:49.883219 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:49.902202 25712 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:49.903687 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:49.902275 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.859870 28944 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.905404 25710 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 4: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:49.907927 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.865937 28698 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:49.914060 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.911388 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.915642 25786 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:49.916158 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.916455 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.916747 25784 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:49.917011 25782 raft_consensus.cc:2376] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6. Re-sending same reply.
I20260709 12:18:49.922417 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.781349 25785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:49.877027 28806 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:18:49.912678 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:49.910452 25779 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:49.943660 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.938836 25785 raft_consensus.cc:606] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:49.944782 25785 raft_consensus.cc:614] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:49.944859 25710 raft_consensus.cc:2410] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:49.912830 25708 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:49.912454 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.912202 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:49.905066 28930 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:49.908659 28970 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:49.951382 25712 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:49.920801 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:49.968453 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.972294 25710 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:49.932111 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.972666 25779 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:49.925280 28698 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:49.973516 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.974510 25786 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:49.971130 25707 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:49.976020 25704 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:49.976891 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.979743 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.980683 25712 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:49.981850 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:49.974748 26335 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:49.983402 28915 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.983778 28899 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:49.983958 28915 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.984486 28899 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.985036 28899 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:49.946369 28845 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.986649 28089 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.998670 28089 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:49.987910 25712 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:49.995687 25708 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:49.996120 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.988435 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:50.006182 25704 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:50.011642 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:50.016790 28089 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:50.024291 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.026381 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:50.027488 25709 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:50.030372 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:49.942703 28757 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:18:50.032284 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:50.061406 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.081902 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:49.976680 28930 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:50.082785 25712 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:50.082979 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:49.986469 28915 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:49.987087 28931 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:49.989182 28970 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:50.090730 25710 raft_consensus.cc:2376] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 5. Re-sending same reply.
I20260709 12:18:50.088783 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:50.092756 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 252285us
I20260709 12:18:50.097847 25710 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:18:50.101717 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:50.102655 25707 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
W20260709 12:18:50.099754 26307 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.641s (SENT) [suppressed 39 similar messages]
I20260709 12:18:50.118854 28994 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:50.119357 28994 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:50.120167 28994 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:50.029428 28845 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:50.128180 25704 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:50.128970 25707 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:50.131948 25707 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:50.132298 28934 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.134582 28934 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:50.136593 28934 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:50.128249 28845 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:50.138900 28922 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.140353 28970 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.142235 28922 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:50.143354 25711 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:18:50.143431 25707 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:50.145154 25707 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:50.146255 25707 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:50.147357 25707 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:18:50.147634 25707 pending_rounds.cc:85] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 5
I20260709 12:18:50.147842 25707 pending_rounds.cc:107] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:18:50.148072 25707 raft_consensus.cc:2889] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:49.710347 25751 leader_election.cc:400] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
W20260709 12:18:50.151119 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 441493us
I20260709 12:18:50.152179 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:50.157950 25710 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:50.158613 25707 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
W20260709 12:18:50.157110 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 277552us
I20260709 12:18:50.162530 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:50.160115 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46554.6us
W20260709 12:18:50.165978 28988 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:18:50.185500 25785 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:50.196338 25785 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:50.170044 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:50.173087 25707 pending_rounds.cc:85] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 6
I20260709 12:18:50.186244 25778 pending_rounds.cc:85] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 1
I20260709 12:18:50.198146 25707 pending_rounds.cc:107] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:18:50.198413 25778 pending_rounds.cc:107] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:18:50.198546 25707 raft_consensus.cc:2889] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:50.198764 25778 raft_consensus.cc:2889] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:50.189520 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:50.191542 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.085632 28787 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:50.198136 25632 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:50.187147 25710 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:50.190868 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:50.185977 25708 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:50.171361 25711 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:50.203814 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.205399 25711 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:18:50.190768 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.192517 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:50.171881 28985 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:50.174204 28989 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:18:50.234449 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.254760 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.313617 26335 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:18:50.173795 28913 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:18:50.166551 28698 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.303296 25784 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:50.319705 25783 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:50.317551 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:50.166909 28965 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
W20260709 12:18:50.312820 25705 consensus_meta.cc:287] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 12:18:50.305838 26341 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:50.333448 25705 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:50.331550 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:50.312574 28938 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:50.320230 28847 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 12:18:50.337038 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.337183 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.318584 28979 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.579s	user 0.000s	sys 0.005s
W20260709 12:18:50.326553 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 135794us
I20260709 12:18:50.217928 28947 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:50.312357 28649 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:18:50.332631 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:50.346859 25708 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:50.324946 25634 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:50.326020 28968 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:50.337625 25636 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:50.339552 25632 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:50.340884 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:50.341953 25710 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:18:50.330104 28990 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:18:50.345995 25629 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:50.363955 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:50.337257 28845 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.365473 28983 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:18:50.368801 28985 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:50.369314 28985 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.366201 28783 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:50.369796 28985 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:50.375198 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:50.382376 28997 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:50.394043 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78849.4us
I20260709 12:18:50.395161 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:50.311165 28930 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.402137 28806 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:50.375371 28922 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.412703 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:50.414736 25751 leader_election.cc:400] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6.
I20260709 12:18:50.415472 25751 leader_election.cc:403] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:50.393038 28845 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.408303 28968 raft_consensus.cc:2764] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:50.421432 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 436) took 1106 ms. Trace:
I20260709 12:18:50.421962 25706 rpcz_store.cc:276] 0709 12:18:49.314988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.377184 (+ 62196us) service_pool.cc:224] Handling call
0709 12:18:50.421232 (+1044048us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54003c91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:50.421363 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371886848}
I20260709 12:18:50.393550 28783 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:50.407833 28757 raft_consensus.cc:2764] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:18:50.419953 28825 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.582s	user 0.005s	sys 0.011s
I20260709 12:18:50.446974 28918 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6.
I20260709 12:18:50.448331 28996 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:50.383164 28997 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:50.449965 28995 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:18:50.418370 28845 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:50.451128 28898 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.000s
I20260709 12:18:50.428588 28974 raft_consensus.cc:2764] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:18:50.477106 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 377399us
I20260709 12:18:50.455600 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:50.488993 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.491973 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.422950 28922 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:50.495643 28922 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:50.499908 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.504204 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.504976 25780 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:50.513639 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:50.513718 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 116993us
I20260709 12:18:50.463204 28825 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:50.502712 28968 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:50.466470 28996 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:50.524915 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:50.516985 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 316081us
I20260709 12:18:50.516822 28787 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:50.536777 26339 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:50.536986 25783 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:50.539583 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.541793 28954 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:50.543639 28974 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:50.543385 28985 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.544938 25784 raft_consensus.cc:2368] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:50.546087 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:50.549182 28996 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:50.549052 28898 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:50.551709 28997 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:50.554692 26307 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:50.554827 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.559608 25786 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:50.522203 28968 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.546062 28985 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:50.551750 25603 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:50.558686 28931 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:50.563218 25783 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:50.567400 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.577488 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:50.581262 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61075.5us
I20260709 12:18:50.582127 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:50.583429 25710 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:50.579530 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:50.608407 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.628484 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.606657 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:50.635221 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:50.569262 28968 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:50.667670 29000 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:50.668664 28900 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:50.670802 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:50.674316 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.580510 28996 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:50.629913 28899 consensus_meta.cc:287] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.644s	user 0.014s	sys 0.000s
I20260709 12:18:50.733536 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.583343 28954 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:50.583832 28898 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:50.571094 28931 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:50.742439 25677 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:50.747345 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 165265us
I20260709 12:18:50.748438 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:50.627327 28997 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:50.758401 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:50.743556 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.759730 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.760512 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:50.761859 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:50.762264 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 91523.5us
I20260709 12:18:50.709096 28985 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:50.722973 28947 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:50.765619 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.745261 28954 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.793649 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:50.762288 28913 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:50.795848 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 460618us
I20260709 12:18:50.796296 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:50.755689 28997 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.796769 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:50.757171 28783 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:50.798564 25601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.006s	sys 0.029s
W20260709 12:18:50.805027 25601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.006s	sys 0.029s
I20260709 12:18:50.761123 28898 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:50.766230 28918 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:50.753993 28938 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:50.762010 28596 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:50.825796 25629 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:50.751864 28899 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.879366 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:50.878310 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:50.806780 28783 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:50.880577 25634 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 4.
I20260709 12:18:50.819504 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.883466 25785 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:18:50.887022 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.887857 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.798564 28913 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:18:50.892452 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 580370us
I20260709 12:18:50.797223 28954 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:50.890717 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 94505.2us
I20260709 12:18:50.894749 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:50.804867 28983 raft_consensus.cc:2764] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:50.807515 28997 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:50.797134 28941 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:18:50.828332 28969 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:50.899950 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:50.906149 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:50.825273 28918 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:50.885033 28783 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:50.914320 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.915807 25785 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:50.825901 28825 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:50.918360 25786 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:18:50.936653 25604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.022s	sys 0.047s
W20260709 12:18:50.951428 25604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.022s	sys 0.047s
I20260709 12:18:50.890312 28985 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:18:50.911336 28969 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:50.914100 28944 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:50.960282 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61251.5us
W20260709 12:18:50.977018 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.914038 28918 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:50.931049 28983 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:50.910140 28596 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 12:18:50.962589 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53202.8us
I20260709 12:18:50.938083 28934 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:50.897219 28931 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.974517 28969 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:51.004640 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.010334 28947 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:51.010784 28947 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:51.011559 28947 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:51.012840 28999 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:18:51.018121 28783 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:51.020944 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2251) took 1049 ms. Trace:
I20260709 12:18:51.021090 28783 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:51.021569 25704 rpcz_store.cc:276] 0709 12:18:49.971163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.147004 (+175841us) service_pool.cc:224] Handling call
0709 12:18:50.147425 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:50.147936 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.147976 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:50.148667 (+   691us) log.cc:844] Serialized 52 byte log entry
0709 12:18:50.523730 (+375063us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:50.559689 (+ 35959us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.561933 (+  2244us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.569108 (+  7175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.569188 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:51.020601 (+451413us) raft_consensus.cc:1685] finished
0709 12:18:51.020631 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:51.020837 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89254,"prepare.queue_time_us":17088,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":17261,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":16762,"spinlock_wait_cycles":15220736,"thread_start_us":374624,"threads_started":1,"wal-append.queue_time_us":765475}
I20260709 12:18:50.980046 28941 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.983450 28825 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:18:51.036671 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50897.6us
I20260709 12:18:51.037829 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:51.055660 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.995826 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.937124 28898 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:50.993687 29005 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:51.025833 28992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.003s	sys 0.056s
W20260709 12:18:51.060573 28698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.081s	sys 0.121s
W20260709 12:18:50.992915 28970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.009s	sys 0.042s
I20260709 12:18:51.086194 25785 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:51.088266 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50563.7us
I20260709 12:18:51.034799 28551 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:50.909077 29000 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:18:51.057821 28931 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:51.098835 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:50.989748 28944 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:51.086887 28992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.004s	sys 0.056s
W20260709 12:18:51.092120 28970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.009s	sys 0.042s
W20260709 12:18:51.087393 28698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.081s	sys 0.121s
W20260709 12:18:51.101969 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 295947us
I20260709 12:18:51.032744 28934 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.092511 29004 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:51.040998 28941 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:51.120958 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:50.994117 28913 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:51.095230 28825 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:51.100304 28596 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:51.103889 29005 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:51.150132 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:51.150950 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 52219.4us
W20260709 12:18:51.130352 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50693.5us
I20260709 12:18:51.137166 25785 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
W20260709 12:18:51.119805 28930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.011s	sys 0.011s
I20260709 12:18:50.994024 28938 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.124785 28899 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:51.163843 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:51.125020 28551 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:51.164759 25786 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:51.123255 29004 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:51.143496 28596 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:51.166131 29002 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:18:51.144042 28970 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:51.179019 25676 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.027s	sys 0.020s
I20260709 12:18:51.181035 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:51.172402 28930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.011s	sys 0.011s
I20260709 12:18:51.152683 29014 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:51.186962 25676 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.028s	sys 0.020s
W20260709 12:18:51.351271 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 976209us
I20260709 12:18:51.187382 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2182) took 1004 ms. Trace:
I20260709 12:18:51.354686 25778 rpcz_store.cc:276] 0709 12:18:50.183257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.185608 (+  2351us) service_pool.cc:224] Handling call
0709 12:18:50.186019 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:50.315771 (+129752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.315829 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:50.337719 (+ 21890us) log.cc:844] Serialized 52 byte log entry
0709 12:18:50.797880 (+460161us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:51.181040 (+383160us) log.cc:844] Serialized 14 byte log entry
0709 12:18:51.187000 (+  5960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.187128 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:51.187181 (+    53us) raft_consensus.cc:1685] finished
0709 12:18:51.187191 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:51.187324 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123804,"raft.queue_time_us":116221,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":8340,"thread_start_us":790537,"threads_started":2,"wal-append.queue_time_us":632094}
W20260709 12:18:51.236197 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.358939 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.242810 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 363508us
I20260709 12:18:51.363233 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:51.364718 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:51.234441 28947 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:51.164366 25533 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:51.237353 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:51.186940 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 381306us
W20260709 12:18:51.215376 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 63382.6us
I20260709 12:18:51.188287 28950 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:51.184804 28596 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:51.202368 25710 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:51.216552 28783 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:51.221045 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:51.228811 28934 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:51.354249 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.508040 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.500150 25704 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:51.492430 25786 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:51.492847 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:51.498592 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 261337us
I20260709 12:18:51.199466 28970 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:51.499751 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121668us
I20260709 12:18:51.484301 29004 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.517844 29004 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:51.508322 28825 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.523275 28551 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:51.525960 25785 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:51.529881 28900 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:51.530542 28900 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:51.531831 29027 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:51.532423 29027 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:51.536537 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.409101 29014 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.513499 29015 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:18:51.538810 29026 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:51.539538 28934 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:51.499879 28783 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.542941 28783 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:51.543627 28944 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:51.410390 29005 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:51.564134 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54196.5us
W20260709 12:18:51.565496 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.565958 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.523763 28551 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:18:51.566398 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.568005 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.603170 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.614646 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 230301us
I20260709 12:18:51.616357 28845 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.617131 28845 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:51.617725 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:51.508250 28847 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:51.651124 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.602977 28551 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:51.602349 28974 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.606871 29021 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.569320 29020 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:51.668423 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.680490 25704 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:51.654184 28847 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:51.682888 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.685642 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.018s	sys 0.004s
W20260709 12:18:51.689956 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.018s	sys 0.004s
W20260709 12:18:51.690886 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 527101us
I20260709 12:18:51.698086 25636 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:51.691314 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:51.626463 28996 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:18:51.665421 28993 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:51.650110 29015 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:51.533351 29027 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:51.686086 25712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.006s	sys 0.047s
I20260709 12:18:51.666715 29020 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.718132 29020 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:51.673550 29021 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:51.732196 28918 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.737s	user 0.009s	sys 0.000s
I20260709 12:18:51.734880 28983 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:51.735635 28983 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:51.736100 28983 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:51.737542 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.738306 25786 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:51.708452 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.709249 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:18:51.712641 25712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.006s	sys 0.047s
W20260709 12:18:51.713613 29019 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:18:51.714653 28825 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:51.692373 28922 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.755545 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.756359 25783 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:51.655907 29024 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:18:51.685160 28965 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:51.752466 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:51.667730 28974 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:51.703852 28993 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:51.739369 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:51.732726 28918 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:51.761003 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 373309us
W20260709 12:18:51.767112 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.769043 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.769923 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.770995 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.786402 25780 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:51.773504 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.787384 25778 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:51.705142 28941 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:51.694973 28698 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:51.799808 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:51.801182 25713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.036s	sys 0.064s
I20260709 12:18:51.752116 28825 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:51.768471 28996 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:51.802372 25713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.037s	sys 0.064s
I20260709 12:18:51.796942 28757 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:18:51.815397 25533 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:18:51.801968 28698 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.790362 28970 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:51.753539 29014 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:51.704600 29022 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:18:51.789767 29010 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:51.831545 29019 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:18:51.832715 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47699.5us
I20260709 12:18:51.785248 28993 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:51.862222 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62536.7us
W20260709 12:18:51.786588 28931 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.679s	user 0.010s	sys 0.003s
I20260709 12:18:51.865869 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:51.866573 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:51.778932 28922 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:51.859808 28954 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.953s	user 0.004s	sys 0.008s
I20260709 12:18:51.792482 28806 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:18:51.850862 29034 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:51.850630 28787 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.820220 29041 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:51.874970 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 171382us
I20260709 12:18:51.876905 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:51.877102 28970 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:51.877703 28938 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.878367 28941 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.878952 29037 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:51.880312 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:51.880199 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:51.882006 25631 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:51.882294 25637 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:51.883452 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:51.863986 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 185767us
I20260709 12:18:51.859441 29024 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:51.852700 28825 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:51.864039 28996 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:51.818403 28965 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.886500 29014 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.874545 29035 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:51.889958 29034 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:51.894150 28997 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.980s	user 0.011s	sys 0.001s
I20260709 12:18:51.923868 25635 raft_consensus.cc:2368] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:51.922818 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:51.887514 29015 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:18:51.855108 28847 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:51.912482 29040 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:51.889423 28954 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.922906 29036 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.967698 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:52.013012 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45436.8us
I20260709 12:18:51.898042 28787 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.967778 28757 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:18:51.936465 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2321) took 1338 ms. Trace:
I20260709 12:18:51.919662 28994 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:51.983911 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:52.034789 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:52.036123 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:52.036280 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:52.037393 25633 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:52.038136 25635 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:51.988477 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.988145 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:51.901278 29010 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:18:52.039496 25778 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:18:51.916404 28970 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.931075 29042 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.041888 29042 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.043314 25635 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:52.044425 29042 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:51.969620 29046 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:51.993950 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:52.019449 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 143527us
I20260709 12:18:51.903612 29041 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:52.045545 25635 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:52.047775 29041 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.027611 29057 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.048437 25630 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:51.972746 28931 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:18:52.049268 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:52.000277 29051 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.984977 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:52.038662 25633 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:18:52.039669 25786 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:51.924533 28941 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.981873 25632 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:52.034245 25709 rpcz_store.cc:276] 0709 12:18:50.598330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.598655 (+   325us) service_pool.cc:224] Handling call
0709 12:18:50.599086 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:50.627525 (+ 28439us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:50.666502 (+ 38977us) log.cc:844] Serialized 14 byte log entry
0709 12:18:51.115353 (+448851us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:51.116065 (+   712us) log.cc:844] Serialized 26 byte log entry
0709 12:18:51.116217 (+   152us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:51.116259 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.116358 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:51.936255 (+819897us) raft_consensus.cc:1685] finished
0709 12:18:51.936279 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:51.936403 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27912,"prepare.queue_time_us":40506,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":71987,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":5814,"thread_start_us":441142,"threads_started":1,"wal-append.queue_time_us":537358}
I20260709 12:18:51.921409 28938 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:51.927292 29037 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:51.991195 29014 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:52.063354 29033 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 12:18:52.064138 29039 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:18:52.064256 28941 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:52.064785 29031 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:18:51.974627 29047 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:18:51.931910 29024 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.988698 28997 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.088671 26339 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:52.049207 28974 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:52.052911 28787 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.000501 29040 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:52.026743 29036 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.115950 29040 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:52.116086 28596 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.001s
I20260709 12:18:52.057809 29046 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.116721 29046 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.117301 29036 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.117871 28997 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.019665 29048 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 12:18:52.095410 26335 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:52.095906 25633 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:52.096968 25629 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:52.059824 28918 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:52.094393 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:52.075770 28947 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.666s	user 0.007s	sys 0.004s
I20260709 12:18:52.048594 29057 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:52.051175 29053 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:52.105775 25634 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:52.107532 29024 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:52.111104 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:52.110602 29049 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:18:52.114197 29032 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:18:52.091957 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.093152 29050 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:52.106213 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 172469us
W20260709 12:18:52.125805 28783 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.002s
I20260709 12:18:52.169006 29050 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:52.169433 28783 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.170140 29050 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.171734 28783 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.172317 28970 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.172528 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:52.173240 25630 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:52.175734 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.118883 28922 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:52.127483 28968 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.371s	user 0.012s	sys 0.000s
I20260709 12:18:52.133551 28993 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.134261 29057 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.134574 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2316) took 1553 ms. Trace:
I20260709 12:18:52.136889 25533 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:52.187625 25712 rpcz_store.cc:276] 0709 12:18:50.581039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.589980 (+  8941us) service_pool.cc:224] Handling call
0709 12:18:50.590408 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:51.103997 (+513589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:51.104057 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:51.127884 (+ 23827us) log.cc:844] Serialized 26 byte log entry
0709 12:18:51.747610 (+619726us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:51.747681 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.747777 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:52.134336 (+386559us) raft_consensus.cc:1685] finished
0709 12:18:52.134356 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.134517 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76552,"raft.queue_time_us":77473,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":1002933,"threads_started":2,"wal-append.queue_time_us":646911}
W20260709 12:18:52.139878 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.143772 25706 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:52.178695 25603 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.144434 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.145141 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.190112 25707 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:52.190068 25711 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:52.146435 25710 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:52.148527 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.150856 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1670) took 1521 ms. Trace:
I20260709 12:18:52.152917 29021 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.154971 29007 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:52.192327 28934 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:52.192705 29007 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:52.192947 28934 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:52.193638 29007 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.193830 28934 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:52.182957 28930 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:18:52.198251 28900 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.160171 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.199626 28900 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:52.199792 26342 raft_consensus.cc:2410] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:52.200696 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.201503 26342 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:52.202313 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.166013 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:52.166289 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.203891 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.205219 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.167788 26343 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:52.168412 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.168644 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.122921 28596 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.177426 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2322) took 1577 ms. Trace:
I20260709 12:18:52.090600 28551 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.178954 26335 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:52.216966 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.217576 29007 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:52.218019 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.182389 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.218779 25704 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:52.066344 29051 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.219511 29021 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:52.221061 29051 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.185299 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:52.221751 28965 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.191810 25708 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:52.222412 29057 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:52.113566 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.222894 29057 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.223312 29057 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:52.224277 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.225335 26337 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:52.226274 26344 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:52.226770 26335 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:52.227315 28954 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.227609 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.228061 26341 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:52.228471 26335 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:52.084398 29034 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:52.229252 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.163409 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.230430 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:52.231700 26339 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:52.236220 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.236307 29014 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.207211 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.237592 26338 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:52.208444 25705 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:52.208124 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:52.186872 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:52.229161 29045 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:52.236512 28551 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.200781 29037 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.191819 25784 rpcz_store.cc:276] 0709 12:18:50.629086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.631231 (+  2145us) service_pool.cc:224] Handling call
0709 12:18:50.723913 (+ 92682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:51.173299 (+449386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:51.173359 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:51.183670 (+ 10311us) log.cc:844] Serialized 26 byte log entry
0709 12:18:51.662763 (+479093us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:51.687991 (+ 25228us) log.cc:844] Serialized 14 byte log entry
0709 12:18:51.688290 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.150379 (+462089us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54000866d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:52.150437 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:52.150509 (+    72us) raft_consensus.cc:1685] finished
0709 12:18:52.150627 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.150788 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5902,"prepare.queue_time_us":25040,"prepare.run_wall_time_us":52,"raft.queue_time_us":19926,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":18051,"spinlock_wait_cycles":407621376,"thread_start_us":706689,"threads_started":2,"wal-append.queue_time_us":650800}
I20260709 12:18:52.245159 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.245899 26342 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:52.211395 25713 rpcz_store.cc:276] 0709 12:18:50.599656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.599798 (+   142us) service_pool.cc:224] Handling call
0709 12:18:50.600150 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:51.104987 (+504837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:51.105034 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:51.126091 (+ 21057us) log.cc:844] Serialized 26 byte log entry
0709 12:18:51.832365 (+706274us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:51.832409 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.832484 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:52.177230 (+344746us) raft_consensus.cc:1685] finished
0709 12:18:52.177250 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.177371 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27356,"raft.queue_time_us":67614,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"thread_start_us":1082252,"threads_started":2,"wal-append.queue_time_us":753366}
I20260709 12:18:52.248952 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.212028 25533 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 4 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:18:52.172498 28938 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.226114 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:52.224287 29004 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.706s	user 0.011s	sys 0.000s
I20260709 12:18:52.108974 28996 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.238814 25633 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:52.280102 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.242306 29051 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.284543 25533 catalog_manager.cc:5697] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 4, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:52.310114 25629 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:52.279207 28992 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.206463 29020 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.313787 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.319682 26340 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:52.314182 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.323084 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.191895 28968 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:52.332665 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 455755us
I20260709 12:18:52.335431 25636 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:52.336786 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:52.292119 29051 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.341406 25779 raft_consensus.cc:2368] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:52.342559 29046 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.344506 29046 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.293366 28825 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.348075 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.380463 25713 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:52.348798 25533 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 5 to 7, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 7 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:18:52.333560 29029 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:52.333288 28922 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.347719 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:52.388990 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:52.396046 25535 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 301) took 33725 ms (client timeout 15000 ms). Trace:
I20260709 12:18:52.407070 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.408890 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:52.412501 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.415000 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.415858 26335 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:52.418411 25635 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:52.422638 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.423113 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:52.423489 25705 raft_consensus.cc:2410] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:52.424424 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.424638 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.426020 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:52.434813 26337 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:52.326028 29006 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:18:52.437064 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:52.348446 28845 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.437340 25635 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:52.320966 28992 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.465979 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.469244 25710 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:52.469401 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.469717 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:52.470096 25705 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:52.470335 25704 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:52.463060 25533 catalog_manager.cc:5697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:52.409060 29046 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:52.345366 28847 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:18:52.388067 29029 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.466027 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:52.459431 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:52.471992 26340 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:52.406700 28922 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:52.476931 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.477785 25710 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 4.
I20260709 12:18:52.479837 29043 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:52.484423 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.010s	sys 0.008s
W20260709 12:18:52.484840 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.010s	sys 0.008s
W20260709 12:18:52.485144 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 579856us
I20260709 12:18:52.458679 28787 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:52.466125 28913 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:52.458025 29015 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:52.438464 28757 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:18:52.472766 29029 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:52.512135 25533 catalog_manager.cc:5697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:18:52.496640 29071 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:18:52.499694 28913 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.497416 29020 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.522110 25533 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:52.493631 29060 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:52.525223 25704 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:52.557701 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:52.527526 25533 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 3 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:52.529166 29063 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:52.493855 29059 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:52.532052 29073 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:18:52.396592 25535 rpcz_store.cc:269] 0709 12:18:18.486481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.486716 (+   235us) service_pool.cc:224] Handling call
0709 12:18:52.189045 (+33702329us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:46.558477 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:46.642758 (+ 84281us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:46.642814 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:18:46.642856 (+    42us) tablet.cc:662] Decoding operations
0709 12:18:46.877195 (+234339us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:46.877322 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:46.877386 (+    64us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:18:46.878343 (+   957us) tablet.cc:701] Row locks acquired
0709 12:18:46.878397 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:18:46.878768 (+   371us) write_op.cc:270] Start()
0709 12:18:46.878981 (+   213us) write_op.cc:276] Timestamp: P: 1783599526878663 usec, L: 0
0709 12:18:46.879039 (+    58us) op_driver.cc:348] REPLICATION: starting
0709 12:18:46.944866 (+ 65827us) log.cc:844] Serialized 2482 byte log entry
0709 12:18:47.851883 (+907017us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:48.543690 (+691807us) op_driver.cc:464] REPLICATION: finished
0709 12:18:48.881674 (+337984us) write_op.cc:301] APPLY: starting
0709 12:18:48.887247 (+  5573us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:48.920660 (+ 33413us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:48.965855 (+ 45195us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:49.217022 (+251167us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:49.217371 (+   349us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:18:49.217423 (+    52us) write_op.cc:312] APPLY: finished
0709 12:18:49.217883 (+   460us) log.cc:844] Serialized 53 byte log entry
0709 12:18:49.218086 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:49.218194 (+   108us) write_op.cc:454] Released schema lock
0709 12:18:49.218261 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:49.263816 (+ 45555us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:49.270071 (+  6255us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2349286,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":266881,"apply.run_cpu_time_us":13975,"apply.run_wall_time_us":362242,"num_ops":6,"prepare.queue_time_us":125838,"prepare.run_cpu_time_us":34960,"prepare.run_wall_time_us":876825,"raft.queue_time_us":547703,"raft.run_cpu_time_us":18958,"raft.run_wall_time_us":308325,"replication_time_us":1592578,"spinlock_wait_cycles":5082880,"thread_start_us":953762,"threads_started":4,"wal-append.queue_time_us":560983,"wal-append.run_cpu_time_us":64916,"wal-append.run_wall_time_us":2803185}]]}
I20260709 12:18:52.506564 28992 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.536804 29041 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:52.540316 26342 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:52.542438 28918 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:52.552811 28922 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:52.555058 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:52.559239 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 80000.9us
I20260709 12:18:52.559337 25533 catalog_manager.cc:5697] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 1 to 3, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:18:52.559423 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.768157 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:52.768661 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 209306us
I20260709 12:18:52.768901 25704 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:18:52.769690 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.771082 25603 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:52.527944 28969 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.487s	user 0.015s	sys 0.000s
W20260709 12:18:52.757357 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 198969us
I20260709 12:18:52.775494 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.776182 25711 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:52.527108 28913 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.777000 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:52.777735 25711 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:52.779143 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:52.640892 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.757812 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.785677 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.786600 25533 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:18:52.793332 25533 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 3, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:52.794706 29040 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:52.787216 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:52.564599 29025 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:52.796608 25711 pending_rounds.cc:85] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 8
I20260709 12:18:52.564693 29059 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.810873 25711 pending_rounds.cc:107] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 12:18:52.798799 25713 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:18:52.811250 25711 raft_consensus.cc:2889] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:52.797238 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.806803 25704 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
W20260709 12:18:52.808139 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 253007us
I20260709 12:18:52.809782 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:52.507329 29015 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.580572 29041 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:52.814992 25603 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:52.778179 29006 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:52.787626 28934 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:52.830731 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.831055 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:52.852674 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:52.853323 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:52.755554 28551 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.245s
I20260709 12:18:52.852424 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:52.797350 29022 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:52.808810 29027 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:52.774601 28985 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:18:52.797209 28969 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:52.854885 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:52.823132 28968 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:52.818542 29025 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:52.813408 29059 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:52.852429 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:52.853361 25780 raft_consensus.cc:2376] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8. Re-sending same reply.
W20260709 12:18:52.859632 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44708.4us
W20260709 12:18:53.514803 25713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.009s	sys 0.008s
W20260709 12:18:53.435098 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.006s	sys 0.004s
W20260709 12:18:53.732194 25713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.010s	sys 0.016s
W20260709 12:18:53.732583 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.266s	user 0.009s	sys 0.004s
I20260709 12:18:52.854157 25785 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:18:53.435359 26307 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.001s	sys 0.009s
W20260709 12:18:53.734568 26307 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.002s	sys 0.011s
W20260709 12:18:53.159246 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 360812us
W20260709 12:18:53.735013 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 968963us
I20260709 12:18:53.735790 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:53.736586 25783 raft_consensus.cc:2376] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6. Re-sending same reply.
W20260709 12:18:53.435513 26310 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.004s	sys 0.005s
W20260709 12:18:53.737418 26310 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.006s	sys 0.006s
W20260709 12:18:53.222767 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.738525 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.858726 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:53.739327 25778 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:52.903357 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 71441us
W20260709 12:18:53.742038 25751 leader_election.cc:336] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.236s (SENT)
W20260709 12:18:53.434824 25711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.007s	sys 0.002s
W20260709 12:18:53.743748 25711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.007s	sys 0.005s
I20260709 12:18:52.854082 25784 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: [8.9-10.10]
W20260709 12:18:53.733109 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 1.29243e+06us
W20260709 12:18:53.435204 25710 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.000s	sys 0.010s
W20260709 12:18:53.746729 25710 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.243s	user 0.003s	sys 0.010s
I20260709 12:18:53.742614 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:53.750999 25633 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.000s
I20260709 12:18:53.768510 25633 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:53.769903 25633 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:53.770066 25704 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:18:53.770838 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2273) took 1217 ms. Trace:
I20260709 12:18:53.771169 25704 rpcz_store.cc:276] 0709 12:18:52.553362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.769416 (+1216054us) service_pool.cc:224] Handling call
0709 12:18:53.769830 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.770587 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.770781 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:53.772315 25676 connection.cc:256] RPC call timeout handler was delayed by -1.9170730830001048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
I20260709 12:18:52.828500 29004 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:53.340139 29050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.004s	sys 0.002s
I20260709 12:18:52.858896 29051 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:52.839551 29006 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:53.777411 25676 connection.cc:256] RPC call timeout handler was delayed by -1.9069879569999557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33140000000000003s for a response
W20260709 12:18:52.895803 29072 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:18:52.851653 28934 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.832983 29073 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:52.894889 29074 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:18:53.787179 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:52.915493 29024 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.000s
W20260709 12:18:52.894099 28900 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.002s
I20260709 12:18:52.893692 28939 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:52.924460 29066 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 12:18:53.746201 29027 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:52.892611 29055 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
W20260709 12:18:53.750376 28757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.008s
I20260709 12:18:52.873458 28970 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:53.785282 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.788913 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:52.858954 28825 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:52.896683 28993 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:53.823769 26307 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:53.800442 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 58439.8us
W20260709 12:18:52.905300 29057 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.682s	user 0.010s	sys 0.000s
I20260709 12:18:53.782943 29051 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.807397 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:53.782430 29006 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:53.812131 25778 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:53.855662 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 522) took 1027 ms. Trace:
W20260709 12:18:53.806551 28918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.013s	sys 0.018s
I20260709 12:18:52.880007 28974 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.919s
I20260709 12:18:53.787573 28934 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:53.825634 26343 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:53.754441 29022 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:53.842947 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 735) took 1674 ms. Trace:
I20260709 12:18:53.910946 26341 rpcz_store.cc:276] 0709 12:18:52.168928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.230229 (+ 61301us) service_pool.cc:224] Handling call
0709 12:18:52.239378 (+  9149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.239708 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.239943 (+   235us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:52.240373 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:18:52.472003 (+231630us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:52.472049 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.472118 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:53.842593 (+1370475us) raft_consensus.cc:1685] finished
0709 12:18:53.842628 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.842853 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":9833,"thread_start_us":177978,"threads_started":1,"wal-append.queue_time_us":250165}
W20260709 12:18:53.912453 28913 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.098s	user 0.010s	sys 0.004s
I20260709 12:18:53.803894 28939 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:53.820995 28757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.007s	sys 0.012s
I20260709 12:18:53.853361 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:53.912889 28913 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:53.913162 28939 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:53.853595 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 531) took 1021 ms. Trace:
I20260709 12:18:53.776413 29004 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:53.856127 25778 rpcz_store.cc:276] 0709 12:18:52.828497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.828647 (+   150us) service_pool.cc:224] Handling call
0709 12:18:53.855603 (+1026956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.913964 25705 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:53.814270 28996 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:53.914054 25782 rpcz_store.cc:276] 0709 12:18:52.832104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.788224 (+956120us) service_pool.cc:224] Handling call
0709 12:18:53.853535 (+ 65311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.915177 25778 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:18:53.914876 28996 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:53.915782 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 528) took 1085 ms. Trace:
I20260709 12:18:53.916123 25778 rpcz_store.cc:276] 0709 12:18:52.830252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.914670 (+1084418us) service_pool.cc:224] Handling call
0709 12:18:53.914971 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.915555 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.915732 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.917248 28996 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:53.794148 28968 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:18:53.914168 29004 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:53.857028 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:53.822788 28970 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.918323 28970 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:53.827061 28900 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.869609 29041 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:53.799456 29073 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:53.930768 28783 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:53.931689 28783 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.932323 29073 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:53.932197 28783 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:53.933869 29073 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.850996 29057 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:53.908999 26307 leader_election.cc:336] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.557s (SENT)
W20260709 12:18:53.857749 28918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.013s	sys 0.018s
I20260709 12:18:53.804340 29024 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:53.817768 29089 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:53.917371 25778 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:53.918360 25786 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:53.820600 29015 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:53.857462 28825 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:53.951460 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 535) took 1117 ms. Trace:
I20260709 12:18:53.951676 25712 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:53.952150 25778 rpcz_store.cc:276] 0709 12:18:52.834147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.916842 (+1082695us) service_pool.cc:224] Handling call
0709 12:18:53.917098 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.951213 (+ 34115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.951406 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.814837 29027 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:53.942411 29041 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:53.865177 29029 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.336s	user 0.011s	sys 0.003s
I20260709 12:18:53.919063 28900 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:53.926651 29025 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:53.947291 29098 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.866891 29051 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:53.970458 26337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 746) took 1635 ms (client timeout 1637 ms). Trace:
W20260709 12:18:53.970925 26337 rpcz_store.cc:269] 0709 12:18:52.334612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.334743 (+   131us) service_pool.cc:224] Handling call
0709 12:18:53.970177 (+1635434us) spinlock_profiling.cc:243] Waited 88.5 ms on lock 0x7b540061d0d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:53.970379 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203625728}
I20260709 12:18:53.972252 29051 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:53.974592 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2267) took 1422 ms. Trace:
I20260709 12:18:53.975004 25711 rpcz_store.cc:276] 0709 12:18:52.551851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.795942 (+244091us) service_pool.cc:224] Handling call
0709 12:18:52.796332 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.829234 (+ 32902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.829264 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:52.829715 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:18:53.744188 (+914473us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:53.744248 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.744362 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:53.974322 (+229960us) raft_consensus.cc:1685] finished
0709 12:18:53.974347 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.974523 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":866213,"threads_started":1,"wal-append.queue_time_us":982456}
I20260709 12:18:53.955444 29024 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:53.934350 29073 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:53.961905 25786 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:53.937183 28783 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:53.964985 26307 leader_election.cc:336] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.641s (SENT)
I20260709 12:18:53.967483 28900 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:53.967999 28950 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:53.979115 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 529) took 1148 ms. Trace:
I20260709 12:18:53.983315 25786 rpcz_store.cc:276] 0709 12:18:52.830464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.827216 (+996752us) service_pool.cc:224] Handling call
0709 12:18:53.979052 (+151836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.983079 28950 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:53.987959 29101 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:53.988314 28950 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:53.989387 28950 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:53.992447 29040 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:53.993016 29040 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:53.998046 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.977340 29098 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:53.998963 26336 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:53.999377 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.000592 29098 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:54.000592 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.001446 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:54.002390 25633 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:18:54.006506 29094 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:18:53.988521 29101 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.009421 29101 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:54.009688 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.000538 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 725) took 1904 ms. Trace:
I20260709 12:18:54.015017 26336 rpcz_store.cc:276] 0709 12:18:52.096517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.096653 (+   136us) service_pool.cc:224] Handling call
0709 12:18:52.110087 (+ 13434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.998811 (+1888724us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b54005a0b20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5906dc0 00007f29b59911c0 00007f29b5992231 00007f29b598ed8f 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c
0709 12:18:53.999428 (+   617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.000299 (+   871us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54005a0354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.000479 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577107584}
I20260709 12:18:54.025830 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:53.964846 29029 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.038347 29059 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.040383 28970 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:54.041301 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.041446 29059 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.044091 25786 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:54.045280 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.045641 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:54.044167 29096 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.046628 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:54.047688 25711 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:54.052135 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.053081 25786 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:54.053186 29096 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.055665 29096 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.056541 29073 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:54.057111 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 584) took 1655 ms. Trace:
I20260709 12:18:54.057647 26342 rpcz_store.cc:276] 0709 12:18:52.401828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.424316 (+ 22488us) service_pool.cc:224] Handling call
0709 12:18:54.056919 (+1632603us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b54005bb654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:54.057038 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140520704}
I20260709 12:18:54.067149 28698 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:53.962414 29089 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.067373 25778 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:54.191973 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:54.193359 26307 leader_election.cc:336] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 2.211s (SENT)
I20260709 12:18:54.157320 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2350) took 1671 ms. Trace:
I20260709 12:18:54.194252 25710 rpcz_store.cc:276] 0709 12:18:52.485823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.486061 (+   238us) service_pool.cc:224] Handling call
0709 12:18:52.486399 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.486671 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:53.748381 (+1261710us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.156976 (+408595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:54.157028 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:54.157074 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.157138 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.157258 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":744,"prepare.queue_time_us":396466,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":145093,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":46,"thread_start_us":1652701,"threads_started":3,"wal-append.queue_time_us":182537}
W20260709 12:18:54.197654 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.197402 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:54.198258 25778 raft_consensus.cc:2376] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6. Re-sending same reply.
I20260709 12:18:54.199177 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.199753 25778 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:18:54.184128 26336 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:18:54.200577 26336 raft_consensus.cc:606] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:54.201182 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.165308 28950 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.085206 28970 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.203933 28950 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:54.134248 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 2.10205e+06us
I20260709 12:18:54.235440 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.236296 26342 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:54.237672 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 505779us
W20260709 12:18:54.134552 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 356312us
I20260709 12:18:54.205865 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.205596 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2268) took 1653 ms. Trace:
I20260709 12:18:54.151178 29096 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.042327 29059 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.239857 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.240479 25713 rpcz_store.cc:276] 0709 12:18:52.552113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.811736 (+259623us) service_pool.cc:224] Handling call
0709 12:18:52.812027 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.814279 (+  2252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.814314 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:52.814765 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:18:53.732818 (+918053us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:53.775264 (+ 42446us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.205251 (+429987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.205343 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:54.205389 (+    46us) raft_consensus.cc:1685] finished
0709 12:18:54.205402 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.205527 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1919,"prepare.queue_time_us":53874,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":236820,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":10101504,"thread_start_us":1287032,"threads_started":2,"wal-append.queue_time_us":969354}
I20260709 12:18:54.179957 29004 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:54.247074 29091 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:18:54.085767 28913 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.249128 28950 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.249750 29006 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.227766 25711 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:54.178081 29029 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.117586 29098 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.271499 29040 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.271636 29098 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.273839 29040 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.274048 29098 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.165241 28985 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:54.165729 29057 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.244812 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.244923 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:54.124729 28939 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.251343 29022 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:54.251911 29059 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.252899 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.281934 25786 raft_consensus.cc:2368] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:54.192281 29051 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:54.255337 28806 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:54.240509 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:54.257619 25637 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.125654 28783 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.263062 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.163698 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:54.268841 29090 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.284420 29090 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.263732 29096 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:54.287729 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.288654 29096 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:54.289211 29096 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.289717 28806 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:54.290675 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.291183 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 524) took 1462 ms. Trace:
I20260709 12:18:54.290764 28806 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.291949 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.249073 29004 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.292651 25780 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:54.275835 25778 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:54.291533 25781 rpcz_store.cc:276] 0709 12:18:52.829132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.829287 (+   155us) service_pool.cc:224] Handling call
0709 12:18:52.855326 (+ 26039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.934711 (+1079385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:53.934757 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:53.934806 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:53.934838 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.290958 (+356120us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400258ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.291005 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.291127 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2481869696}
I20260709 12:18:54.276839 26336 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:54.278823 25629 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:54.280385 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.282819 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.283131 26342 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:54.283723 29059 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.292035 28806 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:54.293140 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2348) took 1818 ms. Trace:
W20260709 12:18:54.269986 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.289417 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:54.283439 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 119852us
W20260709 12:18:54.276119 25601 connection.cc:256] RPC call timeout handler was delayed by -2.0329871469998579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
I20260709 12:18:54.301081 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.301225 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:54.307857 25785 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:54.303115 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 691) took 2424 ms. Trace:
I20260709 12:18:54.309811 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.309568 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.304384 25637 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:54.305078 25786 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:54.307103 25676 connection.cc:256] RPC call timeout handler was delayed by -1.6506730830001288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
I20260709 12:18:54.308171 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:54.311009 25785 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
W20260709 12:18:54.311280 25676 connection.cc:256] RPC call timeout handler was delayed by -1.647073083000123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
I20260709 12:18:54.305645 25705 rpcz_store.cc:276] 0709 12:18:52.474809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.474942 (+   133us) service_pool.cc:224] Handling call
0709 12:18:52.498824 (+ 23882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.914598 (+1415774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.292924 (+378326us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400073ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.293083 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3253994240}
W20260709 12:18:54.307041 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 68368.4us
I20260709 12:18:54.309470 25634 rpcz_store.cc:276] 0709 12:18:51.878224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.878386 (+   162us) service_pool.cc:224] Handling call
0709 12:18:51.878761 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.123682 (+244921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.302861 (+2179179us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540021f054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.303032 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521252608}
I20260709 12:18:54.314406 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:54.316951 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.317137 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.318123 25781 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:54.319077 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.319703 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:54.321425 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:54.323211 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:54.324388 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.289556 29096 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.324949 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.328630 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:54.329151 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.329474 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.330112 25786 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:54.331027 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.331700 25786 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:54.304381 28551 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:54.348112 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.348485 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.336875 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.349807 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.351130 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.349180 26337 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:54.313659 29101 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.335476 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.337205 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.337441 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.338335 29103 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:54.355657 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.356333 25786 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:54.356201 25780 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:18:54.328506 29041 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:54.329928 29040 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.330302 29096 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.340212 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.340570 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.334856 29111 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.341290 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.362131 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.363344 26340 raft_consensus.cc:2368] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8.
I20260709 12:18:54.313126 29007 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.366807 29007 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.357213 28970 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.369184 28970 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.371167 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:54.371773 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.372390 28698 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:54.372946 28698 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.373423 28596 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.375252 28698 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.341784 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.376320 29118 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:54.376628 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.342401 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.376952 29118 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.377856 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:54.316538 29000 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.378794 29111 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.321508 29004 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:54.380281 25637 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:54.380373 29089 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.274845 28985 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.324654 29022 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.357445 26335 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:54.360735 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.349627 29098 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:54.325608 29090 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.362855 29040 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:54.368584 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:54.377890 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.321353 29057 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:54.352502 28551 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.343030 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:54.306742 28783 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.379356 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.360600 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.382905 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:54.335565 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:54.392791 29062 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.889s	user 0.001s	sys 0.003s
I20260709 12:18:54.330127 29092 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:18:54.326678 28931 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:18:54.383553 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:54.472760 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.497004 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.463028 25780 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:54.522626 25601 connection.cc:256] RPC call timeout handler was delayed by -1.0047489809999206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 12:18:54.523307 26341 raft_consensus.cc:2410] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:54.395071 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2343) took 1978 ms. Trace:
W20260709 12:18:54.523442 25601 connection.cc:256] RPC call timeout handler was delayed by -1.0047489809999206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:18:54.442297 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.451680 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.526849 26336 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:54.407886 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:54.529160 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 145667us
I20260709 12:18:54.526656 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.524444 25785 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:54.521471 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.532426 26335 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:18:54.525239 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 189583us
I20260709 12:18:54.477809 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:54.534826 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 166317us
I20260709 12:18:54.523780 25708 rpcz_store.cc:276] 0709 12:18:52.417102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.417242 (+   140us) service_pool.cc:224] Handling call
0709 12:18:52.417618 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.507940 (+ 90322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.507973 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:52.508480 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:18:54.004485 (+1496005us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b180046f7e0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9
0709 12:18:54.006218 (+  1733us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b180046f7e0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:18:54.116664 (+110446us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:54.116721 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.394649 (+277928us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b54003c8cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.394717 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:54.394780 (+    63us) raft_consensus.cc:1685] finished
0709 12:18:54.394800 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.394991 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5859,"raft.queue_time_us":3583,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":14240,"spinlock_wait_cycles":273384448,"thread_start_us":110207,"threads_started":1,"wal-append.queue_time_us":1682052}
I20260709 12:18:54.535805 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:54.536298 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.536912 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:54.537206 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.540076 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.371253 28970 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.539736 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.543215 25785 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:54.545050 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.555191 25786 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:54.379125 29118 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.542829 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:54.553695 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.557914 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:54.559255 26340 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:18:54.560693 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.563723 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.563723 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:54.465905 29098 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.566730 25706 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:54.565912 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.572808 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.573421 25785 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:54.566051 26335 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:18:54.394073 28985 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.523412 29040 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.614398 28983 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:54.542521 29115 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:54.361083 29101 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.575167 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:54.575408 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.464560 28965 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:18:54.528923 29004 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:54.542351 29022 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:54.655678 29022 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.657078 29022 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.578045 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:54.658916 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.379132 29000 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.578392 29006 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.578568 28939 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.582926 28994 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:54.535571 28941 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:54.583715 29036 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.584091 29102 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:54.547844 28596 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:18:54.556772 29035 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:54.562953 29045 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.584946 25706 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:54.675905 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:54.676718 25706 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:54.564342 29105 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
W20260709 12:18:54.589661 29108 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 12:18:54.591027 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:54.592262 29121 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
I20260709 12:18:54.593191 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2327) took 2451 ms. Trace:
I20260709 12:18:54.597896 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.679669 26337 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:54.612540 25676 leader_election.cc:400] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
I20260709 12:18:54.680173 25676 leader_election.cc:403] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:54.535475 29103 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:54.682075 28783 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:54.668578 29027 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.000s
I20260709 12:18:54.682129 29101 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.683276 29027 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.685931 29027 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.570051 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:54.686693 28939 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:54.687048 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 117085us
I20260709 12:18:54.689321 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:18:54.617374 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.575970 26335 pending_rounds.cc:85] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 5
I20260709 12:18:54.692512 26335 pending_rounds.cc:107] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:18:54.692934 26335 raft_consensus.cc:2889] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:54.642145 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.694933 28596 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:54.695545 28596 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:54.695969 28994 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:54.654134 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.464685 29109 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.252s
I20260709 12:18:54.699079 29109 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:54.699222 26341 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 4: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:54.699492 28551 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.700335 26344 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:54.700614 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.701454 29000 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.616955 29115 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:54.702451 29115 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:54.706758 29098 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.605841 29096 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.711458 29111 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.616255 28983 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.713114 29096 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.713023 28983 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.714377 25781 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:54.714707 28983 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:54.611027 26338 raft_consensus.cc:2410] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:54.616531 29089 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:18:54.719169 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.720503 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:54.721474 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1664) took 4200 ms (client timeout 2062 ms). Trace:
I20260709 12:18:54.469862 28934 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:54.722221 25779 rpcz_store.cc:269] 0709 12:18:50.520892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.521057 (+   165us) service_pool.cc:224] Handling call
0709 12:18:54.721159 (+4200102us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54001248d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:54.721353 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4141739264}
I20260709 12:18:54.722879 29006 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.721625 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.723481 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.723636 29006 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.723704 25779 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:54.724052 29006 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.724285 26337 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:54.724514 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.724922 25779 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:54.725147 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.725442 25707 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:54.725281 25779 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.726729 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2262) took 2176 ms. Trace:
I20260709 12:18:54.726977 29118 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:54.725245 26336 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:54.727586 29118 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.728089 29118 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:54.726658 25779 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.729274 25779 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:54.730681 28965 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:54.675194 29124 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:18:54.731278 28965 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.732164 28918 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:54.732563 28918 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.733490 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 532) took 1900 ms. Trace:
I20260709 12:18:54.733680 28965 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.734122 25782 rpcz_store.cc:276] 0709 12:18:52.833377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.914959 (+1081582us) service_pool.cc:224] Handling call
0709 12:18:53.915264 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:54.277615 (+362351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.277637 (+    22us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:54.281051 (+  3414us) log.cc:844] Serialized 52 byte log entry
0709 12:18:54.359130 (+ 78079us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:54.359429 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.359809 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.362174 (+  2365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.362224 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:54.733302 (+371078us) raft_consensus.cc:1685] finished
0709 12:18:54.733321 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.733438 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83,"prepare.queue_time_us":350,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":86,"raft.queue_time_us":45444,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":36935,"spinlock_wait_cycles":18546432,"thread_start_us":407861,"threads_started":2,"wal-append.queue_time_us":142495}
I20260709 12:18:54.736236 29048 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:54.736450 25779 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:18:54.736457 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.737222 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 689) took 2865 ms. Trace:
I20260709 12:18:54.653478 25786 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:54.737787 29103 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6.
W20260709 12:18:54.737723 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 125333us
I20260709 12:18:54.727119 25707 rpcz_store.cc:276] 0709 12:18:52.550435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.555689 (+  5254us) service_pool.cc:224] Handling call
0709 12:18:52.556002 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.725956 (+2169954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.726514 (+   558us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b5400073fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.726676 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4988955904}
I20260709 12:18:54.739984 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:54.740716 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:54.741236 25783 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:54.741616 25707 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:54.697409 28994 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.742657 29103 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.743076 29103 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.733417 28918 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.743388 25784 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:54.743397 29103 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.743925 25784 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.736688 29048 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.745505 29048 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:54.737391 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.746937 29048 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:54.750717 29035 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:54.752323 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63093.6us
I20260709 12:18:54.756012 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 706) took 1885 ms. Trace:
I20260709 12:18:54.673359 28941 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.756389 25631 rpcz_store.cc:276] 0709 12:18:52.870621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.767962 (+897341us) service_pool.cc:224] Handling call
0709 12:18:53.768311 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.786436 (+ 18125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:53.786615 (+   179us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:53.817969 (+ 31354us) log.cc:844] Serialized 26 byte log entry
0709 12:18:54.132307 (+314338us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:54.132369 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.132469 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:54.755759 (+623290us) raft_consensus.cc:1685] finished
0709 12:18:54.755783 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.755948 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17241,"raft.queue_time_us":274686,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":247496,"threads_started":1,"wal-append.queue_time_us":448488}
I20260709 12:18:54.668579 29036 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.758894 29036 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.761942 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 702) took 1989 ms. Trace:
I20260709 12:18:54.741374 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:54.764277 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:54.687258 28939 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:54.765048 26337 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:54.762807 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:54.765923 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.679518 25709 rpcz_store.cc:276] 0709 12:18:52.141656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.145482 (+  3826us) service_pool.cc:224] Handling call
0709 12:18:52.145836 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.146068 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.146100 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:52.146651 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:18:52.315817 (+169166us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:52.315871 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.315974 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:54.392155 (+2076181us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:54.592891 (+200736us) raft_consensus.cc:1685] finished
0709 12:18:54.592914 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.593110 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6967040,"thread_start_us":168703,"threads_started":1,"wal-append.queue_time_us":275083}
I20260709 12:18:54.723737 28934 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:54.737650 25636 rpcz_store.cc:276] 0709 12:18:51.871516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.871691 (+   175us) service_pool.cc:224] Handling call
0709 12:18:51.872203 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.336457 (+464254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.737035 (+2400578us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54001d79d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.737168 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064494464}
I20260709 12:18:54.762377 25629 rpcz_store.cc:276] 0709 12:18:52.772975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.773234 (+   259us) service_pool.cc:224] Handling call
0709 12:18:52.773806 (+   572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.294666 (+1520860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.761700 (+467034us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001dc9d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.761876 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3460828416}
I20260709 12:18:54.764993 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:54.696681 28596 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.745360 25784 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.806320 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:54.796706 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:54.811555 25635 raft_consensus.cc:2368] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:54.773713 25783 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:54.812282 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:54.812415 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.813552 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:54.800464 25631 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:54.807513 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:54.815543 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 272768us
I20260709 12:18:54.815994 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.813675 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.736508 25782 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:54.796670 26342 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:18:54.797010 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 250292us
I20260709 12:18:54.796871 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:54.674996 29045 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.854909 25784 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.820498 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:54.834230 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:54.858876 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.699492 29109 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:54.861209 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:54.861616 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:18:54.819593 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 280370us
I20260709 12:18:54.796290 25707 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:54.799134 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:54.863480 25636 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:54.863651 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:54.800937 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.865696 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.867151 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.820242 25635 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:54.868281 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.868616 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.792446 29042 raft_consensus.cc:2764] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:54.795347 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:54.871136 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.833745 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.871860 25709 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:18:54.869572 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:54.874090 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:54.875455 25637 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:18:54.767681 28941 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.872334 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:54.920414 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:54.854415 29043 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:54.883229 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:54.750011 29004 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:54.876526 28992 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:54.858235 29045 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:54.936077 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 52938.8us
I20260709 12:18:54.856786 29101 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:54.820340 28847 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:54.895025 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.860502 29123 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:54.943686 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:54.819119 29035 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:54.820257 29026 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 12:18:54.923007 29000 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:54.929798 25636 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:54.922926 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2263) took 2372 ms. Trace:
I20260709 12:18:54.951678 25712 rpcz_store.cc:276] 0709 12:18:52.550781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.556179 (+  5398us) service_pool.cc:224] Handling call
0709 12:18:52.556454 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.952478 (+1396024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.922636 (+970158us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54003ac854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:54.922850 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3208326400}
I20260709 12:18:54.952986 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:54.953553 25712 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:54.858381 29111 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:18:54.920981 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:54.820897 29105 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 12:18:54.939452 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:54.958097 25707 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:54.935148 25784 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:54.914737 28931 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:54.941648 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69384.9us
I20260709 12:18:54.874593 28551 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:18:54.892246 28783 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:54.892874 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72484.8us
I20260709 12:18:54.966928 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:54.945655 28965 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:54.946203 25710 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:54.914673 29096 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 12:18:54.964154 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.892246 28939 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:54.949664 25601 leader_election.cc:400] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8.
I20260709 12:18:54.971403 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:54.973062 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:54.973521 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:54.951107 28934 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:54.973852 25709 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:54.968457 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:54.982156 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.990708 25712 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:18:55.059139 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.059687 25751 leader_election.cc:400] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:18:55.060117 25751 leader_election.cc:403] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:55.061357 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:55.067691 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:55.057907 25784 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:55.058298 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:55.071094 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:55.074954 25712 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:55.071228 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:55.074967 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:55.083214 26343 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:55.084739 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:55.085569 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:54.948124 29040 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:55.057871 29059 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.001s
I20260709 12:18:55.105058 25712 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:55.107532 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:55.096320 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:55.084443 28847 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:54.947670 28938 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:18:55.107988 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.058050 29035 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:55.127537 25712 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:55.133414 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:55.134187 25712 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:55.109218 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:55.067301 29042 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:55.139408 25706 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:55.068578 28783 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:55.068821 29101 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:55.136468 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:54.943383 29109 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:55.095715 28950 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.140125 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.142052 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:55.143860 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.149859 25713 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:55.150998 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.105767 29057 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:55.057083 29102 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:18:55.101531 28939 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:55.153307 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:55.074064 29092 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:18:55.151671 25713 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:55.165781 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.166631 25713 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:55.168146 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:55.168705 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:55.141083 29042 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:55.171393 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 99026.5us
I20260709 12:18:55.172376 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:55.137530 29047 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:55.170244 25784 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:55.180186 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:55.174896 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:55.180951 26344 raft_consensus.cc:2376] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6. Re-sending same reply.
I20260709 12:18:55.187675 26344 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:55.164815 28992 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:55.127826 29059 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.173681 29042 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:55.144920 29129 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:18:55.167868 29101 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:55.194347 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:55.177505 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.169260 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.176949 29119 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:18:55.196059 28934 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:55.159550 28974 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:18:55.219429 25713 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:55.183264 29047 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:55.225397 29047 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:55.226110 29000 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:55.226894 28938 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:55.229849 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:55.233891 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.231667 25707 raft_consensus.cc:2368] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8.
I20260709 12:18:55.187007 29098 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:55.245260 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:55.245741 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:55.220834 28947 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:18:55.248278 25784 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:55.211053 29057 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.252802 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:55.254191 25713 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:18:55.259522 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 298633us
I20260709 12:18:55.268553 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:55.275704 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:55.276585 25713 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:55.286728 29147 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:55.287415 29147 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:55.288594 29147 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:55.292098 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.292712 29132 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:18:55.294473 29136 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:18:55.295512 29127 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:18:55.299417 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:55.300482 25705 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:55.292104 28947 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:55.303829 29141 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:55.304394 29141 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:55.307973 29140 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:18:55.308759 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 136494us
I20260709 12:18:55.310179 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:55.247641 29098 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.323195 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:55.290231 29147 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:55.366423 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 48320us
I20260709 12:18:55.356817 29098 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:55.265952 29057 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:55.305490 29141 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:55.377579 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 222950us
I20260709 12:18:55.364235 29113 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.384068 25713 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
W20260709 12:18:55.384708 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 288434us
I20260709 12:18:55.384888 25784 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:18:55.364993 28947 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:55.365357 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:55.389235 26308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.006s	sys 0.009s
W20260709 12:18:55.393074 26308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.006s	sys 0.010s
I20260709 12:18:55.368070 25709 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:55.388309 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 536) took 1228 ms. Trace:
W20260709 12:18:55.393286 25704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.013s
W20260709 12:18:55.395113 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 518640us
W20260709 12:18:55.363476 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 53359.8us
I20260709 12:18:55.396176 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:55.395175 25779 rpcz_store.cc:276] 0709 12:18:54.160212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.737354 (+577142us) service_pool.cc:224] Handling call
0709 12:18:54.737639 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:54.737955 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.737981 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:54.762128 (+ 24147us) log.cc:844] Serialized 26 byte log entry
0709 12:18:55.153112 (+390984us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:55.158732 (+  5620us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.387797 (+229065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.387939 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.388004 (+    65us) raft_consensus.cc:1685] finished
0709 12:18:55.388026 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:55.388224 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9064,"prepare.run_wall_time_us":46,"raft.queue_time_us":42493,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":17573,"spinlock_wait_cycles":15113600,"thread_start_us":527158,"threads_started":2,"wal-append.queue_time_us":406774}
I20260709 12:18:55.398169 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:55.395530 25704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.005s	sys 0.013s
I20260709 12:18:55.398798 25713 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:55.379613 29040 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:55.376756 29007 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:55.400285 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:55.400341 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:55.404757 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.397810 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:55.402015 25713 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 12:18:55.405356 25779 raft_consensus.cc:2410] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:55.403005 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:55.408197 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:55.408540 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:55.406399 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.409343 25636 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:18:55.409570 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.410589 25713 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:55.412045 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:55.386169 29112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.013s
I20260709 12:18:55.426093 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:55.427165 25779 raft_consensus.cc:2376] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6. Re-sending same reply.
I20260709 12:18:55.399372 29148 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:55.403681 29060 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.432830 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:55.385923 29146 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:18:55.374974 28992 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.382819 28934 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.399552 29107 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
W20260709 12:18:55.425379 29112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.000s	sys 0.014s
I20260709 12:18:55.414921 29150 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:55.446900 25786 raft_consensus.cc:606] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:55.432245 29148 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.397776 29113 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:55.458033 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53467.3us
I20260709 12:18:55.460971 29148 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:55.462395 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:55.442898 29040 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:55.462385 29112 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:55.465045 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:55.466125 25779 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:55.449867 29118 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.001s
I20260709 12:18:55.437027 29157 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:55.408178 29000 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.473783 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2206) took 1227 ms. Trace:
I20260709 12:18:55.501457 25779 rpcz_store.cc:276] 0709 12:18:54.246094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.448590 (+1202496us) service_pool.cc:224] Handling call
0709 12:18:55.465952 (+ 17362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.473537 (+  7585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.473728 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:55.502166 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:55.450706 29059 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:55.503511 29057 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:55.503914 25779 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:55.504613 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2208) took 1240 ms. Trace:
W20260709 12:18:55.500411 29123 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.006s
I20260709 12:18:55.505378 29123 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.506433 25705 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:55.504978 25779 rpcz_store.cc:276] 0709 12:18:54.264677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.502283 (+1237606us) service_pool.cc:224] Handling call
0709 12:18:55.503565 (+  1282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.504353 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.504543 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:55.491431 29149 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:55.493638 29113 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:55.494732 29118 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.509835 25779 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:55.510124 29137 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.153s	user 0.001s	sys 0.001s
I20260709 12:18:55.510466 29101 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.500185 29168 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:55.484820 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:55.500087 29043 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.001s
I20260709 12:18:55.512370 29043 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.512504 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:55.507293 29146 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.495402 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:55.485716 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52989.4us
I20260709 12:18:55.511034 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2209) took 1243 ms. Trace:
I20260709 12:18:55.514901 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:55.514964 25779 rpcz_store.cc:276] 0709 12:18:54.267731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.509387 (+1241656us) service_pool.cc:224] Handling call
0709 12:18:55.509683 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.510291 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.510963 (+   672us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:55.516500 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2193) took 1471 ms. Trace:
I20260709 12:18:55.518460 25631 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:55.516499 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:55.519928 29159 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:55.514091 29146 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:55.520746 29133 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:18:55.521014 29118 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:55.522341 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2353) took 1134 ms. Trace:
I20260709 12:18:55.522835 25711 rpcz_store.cc:276] 0709 12:18:54.388368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.390681 (+  2313us) service_pool.cc:224] Handling call
0709 12:18:54.391068 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:54.391525 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.391565 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:54.536187 (+144622us) log.cc:844] Serialized 26 byte log entry
0709 12:18:54.916342 (+380155us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:54.916384 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.916461 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.522092 (+605631us) raft_consensus.cc:1685] finished
0709 12:18:55.522112 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:55.522272 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":357,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":333518,"threads_started":1,"wal-append.queue_time_us":600318}
I20260709 12:18:55.525171 25711 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:55.516949 25785 rpcz_store.cc:276] 0709 12:18:54.044945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.670187 (+625242us) service_pool.cc:224] Handling call
0709 12:18:54.670720 (+   533us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:54.671260 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.671301 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:54.672134 (+   833us) log.cc:844] Serialized 52 byte log entry
0709 12:18:55.163480 (+491346us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:55.469815 (+306335us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.485054 (+ 15239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.485170 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.516273 (+ 31103us) raft_consensus.cc:1685] finished
0709 12:18:55.516295 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:55.516435 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9839,"raft.queue_time_us":2565,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5917,"thread_start_us":703857,"threads_started":2,"wal-append.queue_time_us":466509}
W20260709 12:18:55.487759 29006 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.000s
I20260709 12:18:55.533052 29112 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:55.534101 29146 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:55.525818 29113 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:55.533105 29109 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:55.539276 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.539522 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:55.540400 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:55.449579 29160 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:18:55.548230 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:55.551307 25711 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:55.552702 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:55.553260 25629 raft_consensus.cc:2376] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 6. Re-sending same reply.
I20260709 12:18:55.560632 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:55.514626 29040 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.558861 29154 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:55.563376 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:55.560614 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:55.542403 29051 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:55.566458 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:55.576622 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:55.578948 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 835) took 1228 ms. Trace:
I20260709 12:18:55.578840 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:55.581030 26340 rpcz_store.cc:276] 0709 12:18:54.350221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.371508 (+ 21287us) service_pool.cc:224] Handling call
0709 12:18:55.578760 (+1207252us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54005a0fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:55.578885 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429592320}
I20260709 12:18:55.576551 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:55.562870 29101 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:55.568177 29159 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.580828 29059 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:55.580102 29123 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:55.546193 29006 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.564779 29118 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:18:55.605643 25711 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:55.591888 29051 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.616380 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:55.594995 29057 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:55.617100 26340 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:55.596657 29168 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:55.567725 29040 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:55.644116 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.690056 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.733539 29155 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:55.734525 29166 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:55.735335 29156 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:55.735939 29156 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.736428 29156 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:55.739688 29173 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 12:18:55.741379 29122 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.248s	user 0.000s	sys 0.004s
W20260709 12:18:55.752979 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.770128 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.771183 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.785327 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 827) took 1589 ms. Trace:
W20260709 12:18:55.792809 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.786036 26339 rpcz_store.cc:276] 0709 12:18:54.195453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.195592 (+   139us) service_pool.cc:224] Handling call
0709 12:18:54.195917 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:54.196275 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.196302 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:54.234251 (+ 37949us) log.cc:844] Serialized 26 byte log entry
0709 12:18:54.325967 (+ 91716us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:54.325998 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.326047 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.566002 (+1239955us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:55.745694 (+179692us) raft_consensus.cc:1685] finished
0709 12:18:55.745730 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:55.785220 (+ 39490us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":452,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":26873,"spinlock_wait_cycles":84561792,"thread_start_us":50391,"threads_started":1,"wal-append.queue_time_us":236164}
I20260709 12:18:55.799279 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:55.823364 29147 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:55.823887 29103 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.079s	user 0.000s	sys 0.012s
I20260709 12:18:55.824167 29147 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:55.824508 29103 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:55.825439 29048 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.078s	user 0.009s	sys 0.003s
I20260709 12:18:55.826030 25712 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:55.827145 29155 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:18:55.827966 29036 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.069s	user 0.005s	sys 0.007s
I20260709 12:18:55.828286 29155 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.828558 29036 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:55.745042 29115 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.042s	user 0.007s	sys 0.012s
I20260709 12:18:55.837414 29115 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.841885 29040 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:55.845512 29047 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.619s	user 0.018s	sys 0.007s
I20260709 12:18:55.849156 29006 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:55.854149 29174 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.854944 29174 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.856324 25629 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:18:55.823624 28983 consensus_meta.cc:287] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.108s	user 0.008s	sys 0.005s
I20260709 12:18:55.835848 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:55.836202 29170 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:18:55.836771 29022 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:55.854719 29116 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 12:18:55.883047 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.884603 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.888062 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.880393 29174 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:55.899302 29103 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:55.845975 29047 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.902052 29047 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:55.770255 29112 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.905624 26344 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:55.880188 29022 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:55.908268 29171 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:18:55.908773 25783 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:18:55.915346 25779 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:55.925943 25783 pending_rounds.cc:85] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 3
I20260709 12:18:55.926354 25783 pending_rounds.cc:107] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:18:55.927076 25783 raft_consensus.cc:2889] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:55.928540 26309 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.641s (SENT) [suppressed 58 similar messages]
W20260709 12:18:55.904330 29045 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.959s	user 0.015s	sys 0.000s
I20260709 12:18:55.858124 29146 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:55.961336 29146 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:55.964092 29146 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:55.894207 29157 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 12:18:55.966464 29045 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:55.979414 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.924654 29042 consensus_meta.cc:287] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.002s
W20260709 12:18:55.990154 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.016852 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.000417 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 462) took 1465 ms. Trace:
I20260709 12:18:56.001291 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:56.043308 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.044476 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:56.045207 26336 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:56.003432 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1750) took 3764 ms. Trace:
I20260709 12:18:56.006209 29036 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:56.010643 29115 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:56.040694 25750 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:56.045986 25630 rpcz_store.cc:276] 0709 12:18:52.239290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.239404 (+   114us) service_pool.cc:224] Handling call
0709 12:18:52.241628 (+  2224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.253160 (+ 11532us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:52.253476 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.042975 (+1789499us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b18002f5ae0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:54.054186 (+ 11211us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18002f5ae0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a
0709 12:18:54.054653 (+   467us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.241002 (+186349us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.241200 (+   198us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:54.241703 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 12:18:54.241846 (+   143us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:54.241879 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.241946 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.539300 (+1297354us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b18002f5ae0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:18:56.003172 (+463872us) raft_consensus.cc:1685] finished
0709 12:18:56.003200 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.003362 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10352,"prepare.queue_time_us":5127,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":2039,"raft.queue_time_us":12520,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":202,"spinlock_wait_cycles":39058560,"thread_start_us":229550,"threads_started":2,"wal-append.queue_time_us":282440,"wal-append.run_cpu_time_us":2302,"wal-append.run_wall_time_us":1364972}
I20260709 12:18:55.971458 29045 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:56.121743 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.123801 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55441.4us
I20260709 12:18:56.125187 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:18:56.091085 25713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.006s	sys 0.009s
I20260709 12:18:55.919950 29147 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:55.899376 28947 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:56.111469 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:56.147187 26339 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:56.138552 25713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.007s	sys 0.009s
I20260709 12:18:56.043331 25704 rpcz_store.cc:276] 0709 12:18:54.535315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.402622 (+867307us) service_pool.cc:224] Handling call
0709 12:18:55.420678 (+ 18056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.431750 (+ 11072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.431787 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.432700 (+   913us) log.cc:844] Serialized 26 byte log entry
0709 12:18:55.550004 (+117304us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:55.571411 (+ 21407us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.574913 (+  3502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.575048 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.000058 (+425010us) raft_consensus.cc:1685] finished
0709 12:18:56.000103 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.000321 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12122,"prepare.queue_time_us":20944,"raft.queue_time_us":64096,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":35482,"spinlock_wait_cycles":17662208,"thread_start_us":98444,"threads_started":1,"wal-append.queue_time_us":158885}
I20260709 12:18:56.103258 29159 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:56.152598 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.038631 29042 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:56.156049 26309 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.530s (SENT)
I20260709 12:18:56.143260 29115 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:56.137334 29045 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.161917 26338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.062s	sys 0.166s
W20260709 12:18:56.163405 26338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.062s	sys 0.166s
I20260709 12:18:56.149559 29043 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:56.160995 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:56.170603 25709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.004s	sys 0.006s
W20260709 12:18:56.177000 25709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.004s	sys 0.006s
I20260709 12:18:56.172799 28992 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:56.138494 29022 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:56.180347 29109 consensus_meta.cc:287] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.000s
I20260709 12:18:56.178900 29158 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:56.164929 29045 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.193297 28983 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.208240 28983 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.214180 28918 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.178030 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 460) took 1643 ms. Trace:
W20260709 12:18:56.221287 26340 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.002s
I20260709 12:18:56.227203 26340 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:56.224102 25709 rpcz_store.cc:276] 0709 12:18:54.534804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.401300 (+866496us) service_pool.cc:224] Handling call
0709 12:18:55.401957 (+   657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.413624 (+ 11667us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:55.434902 (+ 21278us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.177544 (+742642us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:56.177622 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:56.177686 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.177793 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.177957 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22810,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":109,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":2590,"thread_start_us":716741,"threads_started":1,"wal-append.queue_time_us":767286}
I20260709 12:18:56.222771 25704 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:56.230394 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.241775 26339 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:56.251621 25704 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:56.251995 29098 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.864s	user 0.006s	sys 0.007s
I20260709 12:18:56.252461 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 474) took 1661 ms. Trace:
I20260709 12:18:56.252384 29098 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.252733 29109 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.253793 25709 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:56.253585 29109 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.254802 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:56.258150 26341 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:56.254418 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 475) took 1662 ms. Trace:
I20260709 12:18:56.258737 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 464) took 1712 ms. Trace:
I20260709 12:18:56.258976 25709 rpcz_store.cc:276] 0709 12:18:54.591691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.253140 (+1661449us) service_pool.cc:224] Handling call
0709 12:18:56.253466 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.254153 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.254350 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:56.260154 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.259177 25713 rpcz_store.cc:276] 0709 12:18:54.546186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.411841 (+865655us) service_pool.cc:224] Handling call
0709 12:18:55.412111 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.445198 (+ 33087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.445237 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.448937 (+  3700us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.148068 (+699131us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:56.148126 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.148222 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.258549 (+110327us) raft_consensus.cc:1685] finished
0709 12:18:56.258578 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.258686 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50303,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":4071,"thread_start_us":679739,"threads_started":2,"wal-append.queue_time_us":742439}
I20260709 12:18:56.260929 25709 raft_consensus.cc:2410] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:56.261405 25713 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:56.252854 25704 rpcz_store.cc:276] 0709 12:18:54.591443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.251041 (+1659598us) service_pool.cc:224] Handling call
0709 12:18:56.251427 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.252170 (+   743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.252386 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:56.262187 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2276) took 1652 ms. Trace:
I20260709 12:18:56.262573 25713 rpcz_store.cc:276] 0709 12:18:54.610299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.260970 (+1650671us) service_pool.cc:224] Handling call
0709 12:18:56.261254 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.261959 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.262140 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38656}
W20260709 12:18:56.264469 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.241144 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.002s	sys 0.011s
W20260709 12:18:56.267228 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.003s	sys 0.011s
I20260709 12:18:56.267521 25709 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:56.267839 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 689024us
I20260709 12:18:56.268214 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2283) took 1656 ms. Trace:
I20260709 12:18:56.268672 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2282) took 1656 ms. Trace:
I20260709 12:18:56.268565 25709 rpcz_store.cc:276] 0709 12:18:54.612146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.267051 (+1654905us) service_pool.cc:224] Handling call
0709 12:18:56.267344 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.267915 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.268160 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:56.268910 26336 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
W20260709 12:18:56.271876 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.335265 25704 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:56.338709 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2280) took 1727 ms. Trace:
I20260709 12:18:56.339080 25704 rpcz_store.cc:276] 0709 12:18:54.611398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.262308 (+1650910us) service_pool.cc:224] Handling call
0709 12:18:56.335034 (+ 72726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.338411 (+  3377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.338644 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:56.335161 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 62650.2us
I20260709 12:18:56.286080 25713 rpcz_store.cc:276] 0709 12:18:54.611906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.267844 (+1655938us) service_pool.cc:224] Handling call
0709 12:18:56.268123 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.268335 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.268360 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:56.268396 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:56.268422 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.268467 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.268624 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:56.336447 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.656s (SENT)
I20260709 12:18:56.268597 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:56.343143 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:56.343436 28994 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.402s	user 0.013s	sys 0.019s
W20260709 12:18:56.343894 28994 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.406s	user 0.013s	sys 0.019s
I20260709 12:18:56.345456 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:56.230249 28992 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:56.356854 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 88335.7us
I20260709 12:18:56.357625 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:18:56.359493 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.362260 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:56.353929 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2290) took 1471 ms. Trace:
I20260709 12:18:56.266592 29109 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:56.371835 25713 rpcz_store.cc:276] 0709 12:18:54.882190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.352720 (+1470530us) service_pool.cc:224] Handling call
0709 12:18:56.353175 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.353439 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.353486 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:56.353550 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:56.353581 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.353678 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.353834 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:56.372973 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.373620 25713 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:56.374997 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.383208 29040 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:56.383759 25707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2361) took 1867 ms (client timeout 2140 ms). Trace:
I20260709 12:18:56.383826 29040 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:56.384411 25707 rpcz_store.cc:269] 0709 12:18:54.516525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.074609 (+558084us) service_pool.cc:224] Handling call
0709 12:18:56.383565 (+1308956us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400105254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:56.383699 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350206720}
I20260709 12:18:56.388234 25707 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:56.388837 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2210) took 2117 ms. Trace:
I20260709 12:18:56.389233 25779 rpcz_store.cc:276] 0709 12:18:54.271823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.516594 (+1244771us) service_pool.cc:224] Handling call
0709 12:18:55.546043 (+ 29449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.926186 (+380143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.388613 (+462427us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400191254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:56.388783 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848647040}
I20260709 12:18:56.351491 29098 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:56.376176 29157 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:56.397594 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:56.426777 29157 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:18:56.399163 29148 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.938s	user 0.012s	sys 0.011s
I20260709 12:18:56.427452 29148 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.427816 29157 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.429582 29148 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:56.431066 25784 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:56.411131 29182 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 12:18:56.411515 29179 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:18:56.411922 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 458) took 1877 ms (client timeout 1656 ms). Trace:
W20260709 12:18:56.433437 25706 rpcz_store.cc:269] 0709 12:18:54.534255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.677449 (+143194us) service_pool.cc:224] Handling call
0709 12:18:56.411573 (+1734124us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400104d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:56.411804 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735585536}
I20260709 12:18:56.434162 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 616) took 1040 ms. Trace:
I20260709 12:18:56.375705 28992 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.434640 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.435521 25706 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:56.436576 28992 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:56.414062 29045 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:56.414410 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:56.437893 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.438611 25636 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:18:56.438982 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1759) took 2124 ms. Trace:
I20260709 12:18:56.439512 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.440559 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.440963 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:56.439635 28941 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:56.442104 28698 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:56.382316 29089 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:56.439991 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.442716 29089 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:56.442862 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.415994 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 25) took 1878 ms. Trace:
I20260709 12:18:56.443558 25786 rpcz_store.cc:276] 0709 12:18:54.537952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.653278 (+115326us) service_pool.cc:224] Handling call
0709 12:18:56.415815 (+1762537us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54002b5554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5989853 00007f29ba95760a 00007f29b3edd7d4 00007f29b3edd762 00007f29b3edd691 00007f29b3edd60c 00007f29b3edc8c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:56.415901 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629774080}
I20260709 12:18:56.444844 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:56.387820 29042 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:56.386312 29040 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:56.355628 29159 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 12:18:56.425787 29156 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.002s
I20260709 12:18:56.376705 29183 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:56.426865 25779 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:56.404744 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:56.407609 28970 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.483511 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2094) took 1651 ms. Trace:
I20260709 12:18:56.483973 25631 rpcz_store.cc:276] 0709 12:18:54.830453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.833961 (+  3508us) service_pool.cc:224] Handling call
0709 12:18:56.481271 (+1647310us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b540020b554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:56.481441 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508896000}
I20260709 12:18:56.434901 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:56.485405 26339 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:56.441780 28941 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.443208 29089 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:56.436605 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.487346 25706 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:56.447170 29079 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 3.218s	user 0.009s	sys 0.027s
W20260709 12:18:56.449623 29142 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.041s	user 0.004s	sys 0.007s
I20260709 12:18:56.453408 29141 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:56.454664 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 82218us
I20260709 12:18:56.456104 25634 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:56.459908 25637 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:56.462328 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.413857 29090 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:56.464327 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:56.417658 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 472) took 1826 ms. Trace:
I20260709 12:18:56.505623 25711 rpcz_store.cc:276] 0709 12:18:54.590793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.552083 (+961290us) service_pool.cc:224] Handling call
0709 12:18:55.569543 (+ 17460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.615339 (+ 45796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.417409 (+802070us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b54002229d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:56.417597 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82050944}
W20260709 12:18:56.467298 29144 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.095s	user 0.008s	sys 0.004s
I20260709 12:18:56.434573 26338 rpcz_store.cc:276] 0709 12:18:55.393910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.533737 (+139827us) service_pool.cc:224] Handling call
0709 12:18:55.534158 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.534721 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.534748 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.535250 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.164970 (+629720us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:56.165028 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.165117 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.433965 (+268848us) raft_consensus.cc:1685] finished
0709 12:18:56.433986 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.434104 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":67,"thread_start_us":592540,"threads_started":1,"wal-append.queue_time_us":670884}
I20260709 12:18:56.513684 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:56.469053 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.514621 25630 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:56.470183 28698 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:56.439391 25633 rpcz_store.cc:276] 0709 12:18:54.314941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.538625 (+223684us) service_pool.cc:224] Handling call
0709 12:18:54.571081 (+ 32456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.762879 (+191798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.762933 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:54.763008 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:54.763065 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.438741 (+1675676us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001c66d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:56.438799 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.438928 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440350464}
I20260709 12:18:56.422183 28806 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:56.472261 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.473857 25713 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:56.472661 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:56.482347 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:56.483054 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78469.1us
I20260709 12:18:56.485951 29119 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:56.496865 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.500552 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.500603 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:56.512696 25706 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:56.518332 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:56.519564 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.525614 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:56.526374 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.527200 28941 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:56.527434 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.504456 29090 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:56.528020 25707 raft_consensus.cc:2376] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10. Re-sending same reply.
I20260709 12:18:56.528736 29090 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.529114 28698 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.529912 29090 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:56.530969 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:56.532090 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 700) took 4059 ms. Trace:
I20260709 12:18:56.533138 25630 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:56.533879 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:56.534651 25779 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:56.535108 28806 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:56.535422 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:56.536053 25779 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:56.536723 25636 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:56.537871 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.538522 25635 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:18:56.539290 29148 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:56.532509 25638 rpcz_store.cc:276] 0709 12:18:52.472889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.473707 (+   818us) service_pool.cc:224] Handling call
0709 12:18:52.474133 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:52.475027 (+   894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.475082 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:52.484312 (+  9230us) log.cc:844] Serialized 26 byte log entry
0709 12:18:52.807450 (+323138us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:52.807509 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.807600 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.446522 (+3638922us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:56.531757 (+ 85235us) raft_consensus.cc:1685] finished
0709 12:18:56.531793 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.532029 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7368,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2190,"spinlock_wait_cycles":35447680,"thread_start_us":264357,"threads_started":1,"wal-append.queue_time_us":390069}
I20260709 12:18:56.540324 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:56.540848 25629 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:56.541041 25634 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:56.541220 25711 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:56.495183 28970 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.543795 28970 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:56.544864 29034 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:56.522984 29007 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:18:56.549410 29007 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:56.549870 29007 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:56.551110 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:56.552448 25786 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:56.484032 29156 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.553490 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:56.521466 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:56.522625 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.522696 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:56.554878 25713 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:56.523048 29045 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:18:56.551208 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:56.559055 29186 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:56.554414 29156 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:56.523260 29159 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:56.489266 29131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.385s	user 0.044s	sys 0.033s
I20260709 12:18:56.560235 25786 raft_consensus.cc:2410] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:56.525422 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.592973 25706 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:56.565838 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.594132 25713 raft_consensus.cc:2376] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8. Re-sending same reply.
W20260709 12:18:56.566316 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48037.5us
I20260709 12:18:56.567731 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.596151 25707 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:56.570389 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:56.597743 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:56.599321 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:56.602706 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.603754 25707 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:56.604707 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.606400 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.607226 25713 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:18:56.611706 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.611959 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:56.608229 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:56.612844 25711 raft_consensus.cc:2376] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4. Re-sending same reply.
I20260709 12:18:56.613808 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:56.551308 29145 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:56.618569 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:56.590098 29159 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:56.591570 29131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.490s	user 0.044s	sys 0.034s
I20260709 12:18:56.589596 28992 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:56.592226 29045 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:56.601693 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:56.611340 29060 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:56.589443 29089 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.627175 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2286) took 2014 ms. Trace:
W20260709 12:18:56.627737 25710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2284) took 2015 ms (client timeout 1742 ms). Trace:
I20260709 12:18:56.618525 29145 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.632700 29035 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:56.633330 29105 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:56.605585 29103 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:56.627200 25712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2277) took 2016 ms (client timeout 2205 ms). Trace:
I20260709 12:18:56.605722 29040 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:56.617204 29183 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:56.621160 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:56.664119 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.666685 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 65067.8us
W20260709 12:18:56.667658 25710 rpcz_store.cc:269] 0709 12:18:54.612384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.920037 (+307653us) service_pool.cc:224] Handling call
0709 12:18:56.627565 (+1707528us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b54000730d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:56.627665 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35395840}
I20260709 12:18:56.632393 28970 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:56.670222 28994 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:56.667441 25709 rpcz_store.cc:276] 0709 12:18:54.612867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.270419 (+1657552us) service_pool.cc:224] Handling call
0709 12:18:56.335960 (+ 65541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.336139 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:56.480628 (+144489us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.626818 (+146190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:56.626873 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:56.626918 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.626983 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.627117 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88599,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":38024,"thread_start_us":285048,"threads_started":2}
W20260709 12:18:56.671137 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 111307us
I20260709 12:18:56.637172 29159 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:56.684415 25712 rpcz_store.cc:269] 0709 12:18:54.610610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.139839 (+529229us) service_pool.cc:224] Handling call
0709 12:18:56.626978 (+1487139us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54003aa554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:56.627131 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555537792}
I20260709 12:18:56.620877 29155 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:18:56.667447 29060 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:56.692104 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71049.2us
I20260709 12:18:56.670030 28918 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:56.693465 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:56.668249 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 617) took 1274 ms. Trace:
I20260709 12:18:56.722383 26340 rpcz_store.cc:276] 0709 12:18:55.394137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.255936 (+861799us) service_pool.cc:224] Handling call
0709 12:18:56.256329 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:56.256775 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.256809 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:56.257451 (+   642us) log.cc:844] Serialized 52 byte log entry
0709 12:18:56.340254 (+ 82803us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:56.514145 (+173891us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.515359 (+  1214us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.667951 (+152592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.668032 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.668070 (+    38us) raft_consensus.cc:1685] finished
0709 12:18:56.668081 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.668197 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149583,"prepare.queue_time_us":437237,"prepare.run_cpu_time_us":206,"prepare.run_wall_time_us":908,"raft.queue_time_us":10928,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":71756,"spinlock_wait_cycles":10742272,"thread_start_us":226289,"threads_started":2,"wal-append.queue_time_us":131921}
I20260709 12:18:56.672648 29145 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.671461 29119 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:56.701246 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:18:56.691288 29190 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:56.732690 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 210969us
I20260709 12:18:56.725397 25637 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.721655 25676 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:56.705214 29098 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.699939 28974 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:56.753808 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52645.9us
I20260709 12:18:56.759002 29204 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:56.762230 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2288) took 2148 ms. Trace:
I20260709 12:18:56.706066 29183 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:56.766258 25629 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:56.718315 29115 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:56.726697 29112 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.720857 29186 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:18:56.773537 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52029.3us
I20260709 12:18:56.732558 29156 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.796617 25704 rpcz_store.cc:276] 0709 12:18:54.613350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.339965 (+1726615us) service_pool.cc:224] Handling call
0709 12:18:56.350796 (+ 10831us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.378985 (+ 28189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.379018 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.382392 (+  3374us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.507390 (+124998us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:56.507432 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.507498 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.761975 (+254477us) raft_consensus.cc:1685] finished
0709 12:18:56.762000 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.762158 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23994,"raft.queue_time_us":28394,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":124579,"threads_started":1,"wal-append.queue_time_us":253630}
I20260709 12:18:56.798828 29204 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:56.800863 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:56.798285 29098 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:56.816045 26309 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.768s (SENT)
I20260709 12:18:56.801040 28941 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:56.801964 29193 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:18:56.817216 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.754168 29157 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.803550 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:56.804522 26340 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:56.718554 29060 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:56.818181 26335 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:56.817890 29157 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.821225 29157 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:56.808980 25637 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:18:56.730659 29190 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.811539 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:56.833312 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:56.833505 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:56.814889 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:56.794250 29207 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:56.831415 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:56.806886 29183 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:56.794430 29000 consensus_queue.cc:1059] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:18:56.810657 29109 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.813984 29045 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.796132 28974 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:56.864331 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.843261 29155 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:56.809715 29115 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.855623 29186 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:56.869473 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.823659 29204 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:56.812481 29112 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.840713 28825 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:18:56.823336 29156 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.870198 28941 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:56.865993 29109 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:56.946741 29207 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.870399 26309 leader_election.cc:336] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.907s (SENT)
I20260709 12:18:57.009979 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:56.901731 29186 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:56.970428 29000 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:56.870333 29045 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:56.856539 29209 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:56.898983 29155 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:57.102265 28596 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.102453 29155 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.104581 29155 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:56.868454 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 58972.1us
I20260709 12:18:57.109995 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:57.110605 29205 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 12:18:57.111042 25637 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:57.112154 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:18:57.113091 29199 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
W20260709 12:18:57.127571 29200 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:18:57.132846 29190 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.149358 29098 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.152746 28969 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.159999 29156 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.164409 29109 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.168030 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.029184 29183 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.033238 29213 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:57.174813 29213 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.177688 29213 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.184175 28974 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.029484 29000 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:57.186671 29000 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.188301 29000 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:57.033437 25787 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:56.975683 28941 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:56.969978 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 103082us
I20260709 12:18:57.026381 29186 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:57.038101 29089 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.100839 29209 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:57.202174 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.205128 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.150185 29098 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.196213 29114 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.246371 29115 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:56.845448 29040 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:57.266019 29216 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:57.265801 29174 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:57.324401 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76773.9us
I20260709 12:18:57.330744 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:57.025271 29207 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:57.355137 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.258947 29209 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.264681 29098 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.364939 29207 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.354157 29089 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:18:57.415122 29089 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:57.415455 29089 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:57.416356 29098 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.416230 29089 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.388144 29035 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:57.420727 29035 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:18:57.422226 29145 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.683s	user 0.004s	sys 0.005s
W20260709 12:18:57.368295 29185 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.024s	user 0.005s	sys 0.006s
I20260709 12:18:57.276306 29114 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:57.372041 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1701) took 3163 ms (client timeout 2043 ms). Trace:
I20260709 12:18:57.377642 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:57.479079 25781 rpcz_store.cc:269] 0709 12:18:54.208935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.324702 (+115767us) service_pool.cc:224] Handling call
0709 12:18:57.371825 (+3047123us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400380254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.371950 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893252352}
I20260709 12:18:57.480020 29147 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:57.480225 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 102713us
I20260709 12:18:57.480541 29147 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:57.481698 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:57.378551 28783 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.482117 29147 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.378835 29112 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.380051 29045 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:57.380247 25780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1709) took 3136 ms (client timeout 3115 ms). Trace:
I20260709 12:18:57.483130 28783 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.381177 29034 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.484153 29089 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:57.484486 29034 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.485769 28783 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:57.381687 29184 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.037s	user 0.008s	sys 0.032s
W20260709 12:18:57.483826 25780 rpcz_store.cc:269] 0709 12:18:54.244027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.357129 (+113102us) service_pool.cc:224] Handling call
0709 12:18:57.379934 (+3022805us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400257bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.380145 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185764736}
I20260709 12:18:57.486771 29034 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.381664 29115 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:57.389389 29060 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.571s	user 0.004s	sys 0.004s
I20260709 12:18:57.286154 28941 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:57.392365 29090 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.862s	user 0.014s	sys 0.000s
I20260709 12:18:57.394337 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:57.489002 25782 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:18:57.397526 29159 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.678s	user 0.007s	sys 0.000s
W20260709 12:18:57.401978 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 541) took 3239 ms (client timeout 1968 ms). Trace:
I20260709 12:18:57.402987 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:57.323390 29216 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:57.408499 28931 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:57.411342 29107 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:57.413156 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:57.367470 29209 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.351890 29213 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.424840 29204 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.422032 29035 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.428714 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.504910 26336 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:57.431031 26341 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:57.432081 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.432426 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.506085 26340 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:18:57.432719 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.506593 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.507326 26338 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
I20260709 12:18:57.435540 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 614) took 2042 ms. Trace:
I20260709 12:18:57.434306 29029 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:57.508289 26344 rpcz_store.cc:276] 0709 12:18:55.393348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.393491 (+   143us) service_pool.cc:224] Handling call
0709 12:18:55.393815 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.914713 (+520898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.435289 (+1520576us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b540055a354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:57.435469 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176450432}
W20260709 12:18:57.401296 29141 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.886s	user 0.008s	sys 0.003s
I20260709 12:18:57.444667 28983 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:57.446470 29198 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.644s	user 0.000s	sys 0.010s
W20260709 12:18:57.449879 29178 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.287s	user 0.014s	sys 0.027s
I20260709 12:18:57.451440 29191 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:57.458309 29022 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:57.464349 29164 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.960s	user 0.007s	sys 0.013s
I20260709 12:18:57.487361 29163 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.475512 29186 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.493897 29045 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:18:57.516319 29191 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:57.487705 29145 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.494433 29060 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.606963 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:57.608029 25712 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:57.609921 29060 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.450249 25713 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
W20260709 12:18:57.377722 25603 connection.cc:256] RPC call timeout handler was delayed by -1.4252752009999767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260709 12:18:57.566895 25706 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:18:57.469473 25676 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.003s	sys 0.006s
I20260709 12:18:57.371332 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2359) took 2855 ms. Trace:
W20260709 12:18:57.424327 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.621272 25676 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.004s	sys 0.006s
W20260709 12:18:57.621762 29042 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.010s	sys 0.009s
W20260709 12:18:57.622071 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 767660us
I20260709 12:18:57.623896 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:57.625424 29045 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:57.625672 25709 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:18:57.567117 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 77327.7us
I20260709 12:18:57.461755 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:57.492239 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.627305 25779 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:57.627902 25782 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:57.629793 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.457027 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1694) took 5667 ms. Trace:
W20260709 12:18:57.568475 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 165426us
I20260709 12:18:57.631407 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:57.631042 25787 rpcz_store.cc:276] 0709 12:18:51.792724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.798329 (+  5605us) service_pool.cc:224] Handling call
0709 12:18:51.850047 (+ 51718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.195888 (+5345841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.456543 (+260655us) spinlock_profiling.cc:243] Waited 5.18 s on lock 0x7b54001252d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:57.456884 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11906034432}
W20260709 12:18:57.490463 25778 rpcz_store.cc:269] 0709 12:18:54.162627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.200958 (+ 38331us) service_pool.cc:224] Handling call
0709 12:18:57.401809 (+3200851us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b54001243d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.401909 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166333312}
I20260709 12:18:57.635787 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.636740 25778 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
W20260709 12:18:57.448783 25784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 547) took 2054 ms (client timeout 2156 ms). Trace:
I20260709 12:18:57.569446 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.638554 25782 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:18:57.639151 25780 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:57.639844 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:57.444149 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.621346 25705 rpcz_store.cc:276] 0709 12:18:54.515935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.301091 (+785156us) service_pool.cc:224] Handling call
0709 12:18:55.301440 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.506825 (+205385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.371081 (+1864256us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400203354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:57.371268 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470946176}
I20260709 12:18:57.589277 26343 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:57.589906 26340 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:57.643656 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.461354 26337 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 12:18:57.645376 26337 pending_rounds.cc:85] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 8
I20260709 12:18:57.645864 26337 pending_rounds.cc:107] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 12:18:57.634219 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:57.646207 26337 raft_consensus.cc:2889] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:57.646991 25787 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:57.467633 26342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 602) took 2489 ms (client timeout 2140 ms). Trace:
W20260709 12:18:57.647970 26342 rpcz_store.cc:269] 0709 12:18:54.978272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.978506 (+   234us) service_pool.cc:224] Handling call
0709 12:18:57.467375 (+2488869us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000a37d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.467536 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477329664}
W20260709 12:18:57.622404 29042 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.010s	sys 0.009s
I20260709 12:18:57.652107 29007 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.595890 25704 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:57.499965 28931 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:57.654852 29007 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:57.490142 28918 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:57.512013 28941 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:57.599663 28983 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:57.516600 28992 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.600903 29112 raft_consensus.cc:993] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:18:57.663204 29112 raft_consensus.cc:1081] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:18:57.661669 28992 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.518759 29035 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:57.666770 28983 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:57.668848 28992 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.668524 28983 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.494290 29115 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:57.670940 26309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.009s	sys 0.011s
W20260709 12:18:57.671408 26309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.009s	sys 0.011s
W20260709 12:18:57.672799 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 802375us
I20260709 12:18:57.494824 29159 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.675602 26342 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:57.675959 29159 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.526930 29191 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.677848 25786 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:57.678360 29114 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.679698 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:57.680586 25704 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:57.680871 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:57.681696 25708 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:57.682628 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48344.9us
I20260709 12:18:57.682961 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:57.715992 26340 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:57.523939 29216 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.717643 29216 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:57.529114 29204 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.509519 29029 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:57.521919 29163 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:57.564419 25679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.017s	sys 0.048s
W20260709 12:18:57.567222 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.619112 25603 connection.cc:256] RPC call timeout handler was delayed by -1.2434752009999102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 12:18:57.723098 25603 connection.cc:256] RPC call timeout handler was delayed by -1.2218752010001026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 12:18:57.722813 29029 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:57.722836 25679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.018s	sys 0.048s
I20260709 12:18:57.726171 29004 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:57.726960 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.727377 25630 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:18:57.727118 29004 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.730093 25633 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:57.638866 25784 rpcz_store.cc:269] 0709 12:18:55.394220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.411812 (+ 17592us) service_pool.cc:224] Handling call
0709 12:18:57.448600 (+2036788us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540030af54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.448728 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2341831040}
I20260709 12:18:57.731251 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.731540 29004 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.732121 25784 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:57.640506 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.733816 25780 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:57.734838 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.735388 25780 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:18:57.736217 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.736744 25780 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:57.480365 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.434594 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:57.739594 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 305013us
I20260709 12:18:57.740021 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:57.591078 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.743124 29159 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:57.743882 25781 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:57.745114 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.745473 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:57.745836 25781 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:57.747119 29204 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:57.747064 28918 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:57.747848 29204 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:57.748080 28918 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.748407 29204 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:57.748672 28918 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.742063 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:57.751194 25633 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:57.752457 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:57.753259 25633 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:57.638144 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:57.755039 25778 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:57.755738 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:57.511294 29213 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.781348 29213 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.683341 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:57.783447 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:57.783936 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.784983 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:57.785902 29209 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.786392 25785 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:57.786860 29219 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.787740 29168 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.787884 29219 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.788506 29168 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.789422 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2211) took 3517 ms. Trace:
I20260709 12:18:57.687541 25707 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:57.688089 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.685963 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:57.689424 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.690223 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2204) took 3450 ms. Trace:
I20260709 12:18:57.690330 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.693894 29186 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.699879 25709 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:57.802671 25787 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:18:57.789839 25785 rpcz_store.cc:276] 0709 12:18:54.272134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.526409 (+1254275us) service_pool.cc:224] Handling call
0709 12:18:57.403421 (+1877012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.787113 (+383692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.788570 (+  1457us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54001243d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:57.789253 (+   683us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54001243d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.789367 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5195143296}
I20260709 12:18:57.804409 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:57.702376 29145 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:57.703474 29183 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.007s	sys 0.004s
I20260709 12:18:57.704308 26339 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:57.705798 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.707590 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:57.707896 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.709471 26342 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:57.709636 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.710716 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.711192 26344 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:57.714056 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:57.715065 25636 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
W20260709 12:18:57.723536 25603 connection.cc:256] RPC call timeout handler was delayed by -1.0607752010000695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 12:18:57.621145 29099 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:57.830727 29099 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:57.832095 29099 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.833456 29099 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:57.835016 25780 raft_consensus.cc:2410] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:57.835641 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2219) took 1020 ms. Trace:
I20260709 12:18:57.836035 25780 rpcz_store.cc:276] 0709 12:18:56.815157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.737346 (+922189us) service_pool.cc:224] Handling call
0709 12:18:57.835563 (+ 98217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:57.837190 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.837972 25780 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:57.841329 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:57.743891 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.842351 25785 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:57.842996 25630 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:57.746441 29163 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.830089 25603 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:57.845189 29006 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:57.845721 29006 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:57.847105 29006 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.645990 25782 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:57.656955 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 491) took 1087 ms. Trace:
I20260709 12:18:57.746289 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1731) took 1379 ms. Trace:
I20260709 12:18:57.733237 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.746629 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:57.774186 25712 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:57.762640 29029 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:57.776713 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:57.785089 29090 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.878906 25778 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:57.688716 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1729) took 1323 ms. Trace:
I20260709 12:18:57.791971 25783 rpcz_store.cc:276] 0709 12:18:54.239858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.861096 (+621238us) service_pool.cc:224] Handling call
0709 12:18:54.880319 (+ 19223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.927604 (+1047285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.689601 (+1761997us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400087ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:57.689944 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363986816}
I20260709 12:18:57.850360 25708 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:57.877758 25711 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 5: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:18:57.858526 25781 rpcz_store.cc:276] 0709 12:18:56.366878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.744794 (+1377916us) service_pool.cc:224] Handling call
0709 12:18:57.746240 (+  1446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:57.885386 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.857831 25706 rpcz_store.cc:276] 0709 12:18:56.570001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.612817 (+ 42816us) service_pool.cc:224] Handling call
0709 12:18:56.619832 (+  7015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.620567 (+1000735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.656601 (+ 36034us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b5400235d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:57.656859 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174993280}
I20260709 12:18:57.879557 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2231) took 1013 ms. Trace:
I20260709 12:18:57.889598 25778 rpcz_store.cc:276] 0709 12:18:56.867237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.637643 (+770406us) service_pool.cc:224] Handling call
0709 12:18:57.879455 (+241812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:57.783610 29191 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:57.811367 29168 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:57.898607 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 152168us
I20260709 12:18:57.811094 29219 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:57.881915 25786 rpcz_store.cc:276] 0709 12:18:56.365271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.592433 (+227162us) service_pool.cc:224] Handling call
0709 12:18:57.688420 (+1095987us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400099a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.688617 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432896768}
I20260709 12:18:57.908387 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.896569 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2214) took 1084 ms. Trace:
W20260709 12:18:57.888518 26309 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.791s (SENT)
I20260709 12:18:57.909521 25784 rpcz_store.cc:276] 0709 12:18:56.813215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.732889 (+919674us) service_pool.cc:224] Handling call
0709 12:18:57.896403 (+163514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:57.906878 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:57.918284 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.786566 29209 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:57.926532 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.917524 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:57.886796 29213 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:57.939442 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 148014us
W20260709 12:18:57.943128 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.943876 25603 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:57.941722 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.917500 29114 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.961385 28931 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.962001 29114 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:57.962759 29107 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.963505 29107 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.965098 29114 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.965663 29107 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:57.965932 29040 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.966741 29040 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.967897 29115 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:57.969754 25782 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:57.970124 29043 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.970927 29043 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:57.973609 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82758.3us
I20260709 12:18:57.974140 25783 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:57.975486 29158 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:57.976255 29158 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:57.976953 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:57.962630 25778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:18:57.971163 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:57.966760 25630 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:57.983434 29207 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:57.984198 29207 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:57.991487 29089 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.007s
I20260709 12:18:57.941262 29112 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:58.002847 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 110286us
I20260709 12:18:58.004817 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1723) took 3092 ms. Trace:
I20260709 12:18:58.007927 25783 rpcz_store.cc:276] 0709 12:18:54.902760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.973528 (+3070768us) service_pool.cc:224] Handling call
0709 12:18:57.973881 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.979147 (+  5266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.995259 (+ 16112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:57.999096 25710 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:58.005506 25633 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:58.011842 25709 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:58.004685 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.016382 25785 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:58.016992 25707 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:57.978415 29221 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:58.025986 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:58.035307 25704 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:18:58.037165 25778 raft_consensus.cc:606] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:58.047585 25778 raft_consensus.cc:614] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:58.054618 25750 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:58.057838 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:57.948028 29163 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:58.005334 29112 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.047385 29060 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:18:58.071620 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.072362 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 195822us
I20260709 12:18:58.077610 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 26) took 1541 ms. Trace:
I20260709 12:18:58.079921 25778 rpcz_store.cc:276] 0709 12:18:56.506816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.917604 (+1410788us) service_pool.cc:224] Handling call
0709 12:18:58.048081 (+130477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.085183 29112 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:58.028748 29221 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.938742 29090 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.056726 29219 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.107775 25783 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:58.107069 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:58.110066 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:58.108006 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:58.110947 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.106719 25784 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:58.103919 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 488) took 1330 ms. Trace:
I20260709 12:18:58.130301 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.136366 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 549) took 2730 ms. Trace:
I20260709 12:18:58.136669 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 552) took 2550 ms. Trace:
I20260709 12:18:58.137244 25783 rpcz_store.cc:276] 0709 12:18:55.574046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.023282 (+2449236us) service_pool.cc:224] Handling call
0709 12:18:58.107493 (+ 84211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.118626 (+ 11133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.124496 (+  5870us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.127215 25713 rpcz_store.cc:276] 0709 12:18:56.569166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.607949 (+ 38783us) service_pool.cc:224] Handling call
0709 12:18:57.856806 (+1248857us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54002260d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:57.899316 (+ 42510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922461440}
I20260709 12:18:58.137073 25784 rpcz_store.cc:276] 0709 12:18:55.394703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.998608 (+2603905us) service_pool.cc:224] Handling call
0709 12:18:58.077544 (+ 78936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.124991 (+ 47447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.125198 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.102308 29141 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:58.118988 29221 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:58.103305 29155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.021s	sys 0.002s
I20260709 12:18:58.103824 29004 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:58.145231 26309 leader_election.cc:336] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.950s (SENT)
W20260709 12:18:58.160041 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 122323us
I20260709 12:18:58.052278 29213 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.170722 29045 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:58.183995 28950 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.184660 28950 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:58.186586 29202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.043s	sys 0.050s
W20260709 12:18:58.186993 29202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.094s	user 0.043s	sys 0.050s
I20260709 12:18:58.188265 29103 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.188951 29103 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.142616 29040 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.144785 29168 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.197701 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 553) took 2623 ms. Trace:
I20260709 12:18:58.206045 25785 rpcz_store.cc:276] 0709 12:18:55.574694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.036096 (+2461402us) service_pool.cc:224] Handling call
0709 12:18:58.078782 (+ 42686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.179691 (+100909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.179734 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:58.197419 (+ 17685us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:58.197463 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.197527 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.197650 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.201453 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:58.204704 25779 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.002s
I20260709 12:18:58.207432 25779 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:18:58.107977 29163 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.106650 29060 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.170918 29004 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:58.213485 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.164809 29183 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:58.195549 29141 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:58.221845 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:58.228463 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2230) took 1365 ms (client timeout 1623 ms). Trace:
W20260709 12:18:58.228945 25779 rpcz_store.cc:269] 0709 12:18:56.858454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.461421 (+602967us) service_pool.cc:224] Handling call
0709 12:18:58.219427 (+758006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.231750 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 558) took 2218 ms. Trace:
W20260709 12:18:58.223385 29007 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 12:18:58.232251 25783 rpcz_store.cc:276] 0709 12:18:56.013068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.179840 (+2166772us) service_pool.cc:224] Handling call
0709 12:18:58.215602 (+ 35762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.221834 (+  6232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.221873 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:58.226473 (+  4600us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:58.231460 (+  4987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.231543 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.231674 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.248036 25780 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:58.248823 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 557) took 2240 ms. Trace:
I20260709 12:18:58.249148 25780 rpcz_store.cc:276] 0709 12:18:56.008787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.170968 (+2162181us) service_pool.cc:224] Handling call
0709 12:18:58.228792 (+ 57824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.248530 (+ 19738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.248759 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:58.230079 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 138480us
I20260709 12:18:58.209863 29158 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.220770 29004 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:58.207548 29040 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:58.257074 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.260954 29043 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:58.268929 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.202064 29219 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.232323 29216 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:58.216353 29168 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:58.379710 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.380472 29168 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.302637 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.303025 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:18:58.381997 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.384513 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:58.385123 25784 raft_consensus.cc:2376] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8. Re-sending same reply.
I20260709 12:18:58.319227 29141 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.286180 29183 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.323063 25783 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:58.327756 29040 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.334877 25704 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:18:58.335632 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 199675us
I20260709 12:18:58.342962 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.351508 29090 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.355229 29209 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.356840 29163 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.363890 29060 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.365043 29213 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:58.254354 28950 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:58.367362 29237 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:58.368060 29219 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.294976 28931 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.369010 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.425724 25633 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:18:58.370549 29022 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.770s	user 0.009s	sys 0.007s
I20260709 12:18:58.374895 25781 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:58.428702 26309 leader_election.cc:336] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.038s (SENT)
I20260709 12:18:58.428936 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2217) took 1614 ms. Trace:
I20260709 12:18:58.376626 25779 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:58.429253 25781 rpcz_store.cc:276] 0709 12:18:56.814086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.228052 (+1413966us) service_pool.cc:224] Handling call
0709 12:18:58.374713 (+146661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.428706 (+ 53993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.428887 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.430078 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2224) took 1607 ms. Trace:
I20260709 12:18:58.408854 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.430424 25779 rpcz_store.cc:276] 0709 12:18:56.823003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.286723 (+1463720us) service_pool.cc:224] Handling call
0709 12:18:58.376363 (+ 89640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.429848 (+ 53485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.430031 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.298413 29043 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:58.409155 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 110503us
W20260709 12:18:58.419831 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 225883us
I20260709 12:18:58.427702 25709 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
W20260709 12:18:58.405103 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 116625us
I20260709 12:18:58.400540 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2220) took 1585 ms. Trace:
I20260709 12:18:58.401750 29103 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.445312 25783 rpcz_store.cc:276] 0709 12:18:56.815467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.284375 (+1468908us) service_pool.cc:224] Handling call
0709 12:18:58.322778 (+ 38403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.400211 (+ 77433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.400478 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.433555 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:58.452080 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:58.456187 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:58.456873 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:58.458909 29107 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:58.459503 29107 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:58.444423 28918 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.007s
I20260709 12:18:58.462901 28918 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.465150 28918 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.458940 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:58.458151 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:18:58.475548 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 65717.8us
I20260709 12:18:58.490576 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.491700 25704 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:18:58.462172 29107 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.425433 29209 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:18:58.461690 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:58.502601 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.508090 25635 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:58.447540 29159 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.445722 29207 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.512529 29159 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.538595 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:58.541883 25704 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:58.500316 29043 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.546227 25778 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:18:58.546965 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2226) took 1715 ms. Trace:
I20260709 12:18:58.547272 25778 rpcz_store.cc:276] 0709 12:18:56.831168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.513303 (+1682135us) service_pool.cc:224] Handling call
0709 12:18:58.543302 (+ 29999us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.546683 (+  3381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.546886 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.549486 25781 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:58.550086 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2229) took 1694 ms. Trace:
I20260709 12:18:58.550386 25781 rpcz_store.cc:276] 0709 12:18:56.856177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.534925 (+1678748us) service_pool.cc:224] Handling call
0709 12:18:58.549194 (+ 14269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.549874 (+   680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.550036 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.550940 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:58.551628 25781 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:58.543427 29022 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:18:58.553833 25712 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:18:58.554210 25712 pending_rounds.cc:85] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 7
I20260709 12:18:58.554415 25712 pending_rounds.cc:107] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:18:58.554672 25712 raft_consensus.cc:2889] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:58.537922 29203 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:18:58.543882 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.567104 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.561120 25707 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:58.601811 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.602763 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:58.604784 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.610965 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:18:58.604199 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:58.608187 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.532675 29209 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.614071 25708 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
W20260709 12:18:58.601218 29035 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.766s	user 0.014s	sys 0.000s
I20260709 12:18:58.616585 25783 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:18:58.613500 25706 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:18:58.617368 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:58.623297 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 200281us
I20260709 12:18:58.598439 28950 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.608501 29147 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.643723 29147 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.645383 29203 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.646116 29203 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.646761 29245 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:58.648784 29204 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.900s	user 0.011s	sys 0.005s
I20260709 12:18:58.649504 29203 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.649355 29204 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.652945 28983 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:58.653520 28983 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:18:58.655288 29183 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.656281 28950 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.659605 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.658812 25536 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 9, leader changed from <none> to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 9 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:58.660840 25712 raft_consensus.cc:2410] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:18:58.662135 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.620600 25783 pending_rounds.cc:85] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 5
I20260709 12:18:58.662834 25783 pending_rounds.cc:107] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:18:58.663223 25783 raft_consensus.cc:2889] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:58.633220 29246 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.158s	user 0.005s	sys 0.000s
I20260709 12:18:58.633621 29174 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.681739 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.638844 29213 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:58.639626 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.640436 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.607820 29007 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.642473 29043 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.644255 29040 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:58.663834 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.629618 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 58637.7us
I20260709 12:18:58.669838 25704 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:58.622830 25711 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 4.
W20260709 12:18:58.614665 29191 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.002s
W20260709 12:18:58.635432 29243 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:18:58.683405 25708 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:58.683693 25710 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:58.684820 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:18:58.629254 29099 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.795s	user 0.014s	sys 0.007s
I20260709 12:18:58.695274 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.718585 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:58.687615 25707 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:58.722644 25705 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:18:58.676596 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:58.689700 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:18:58.725595 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49137.3us
I20260709 12:18:58.711916 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:58.711490 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:58.714362 25706 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:18:58.731771 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 188176us
I20260709 12:18:58.732180 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2227) took 1895 ms. Trace:
I20260709 12:18:58.731778 29040 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:58.733235 29204 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.733716 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:58.734333 25676 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:58.735555 29251 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:58.736662 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:58.737634 29251 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:58.686357 29213 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.738696 29174 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.741758 29174 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.733281 29040 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.743031 29040 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:58.743842 29209 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:58.745913 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.746851 28950 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.748050 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:58.749948 25603 leader_election.cc:400] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8.
I20260709 12:18:58.750381 25603 leader_election.cc:403] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:58.723838 29022 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.755295 29022 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.757584 29006 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:58.760123 29006 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:58.718066 28983 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.719810 29007 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.754199 29035 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:58.764695 29035 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:58.764981 29007 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:58.765214 29035 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:58.767071 26337 raft_consensus.cc:2410] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:18:58.769551 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 550) took 3198 ms. Trace:
I20260709 12:18:58.769325 29004 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.732712 25783 rpcz_store.cc:276] 0709 12:18:56.836657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.522291 (+1685634us) service_pool.cc:224] Handling call
0709 12:18:58.559046 (+ 36755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.731845 (+172799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.732110 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.770139 25782 rpcz_store.cc:276] 0709 12:18:55.571542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.984658 (+2413116us) service_pool.cc:224] Handling call
0709 12:18:57.985222 (+   564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:57.986876 (+  1654us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:58.148810 (+161934us) log.cc:844] Serialized 14 byte log entry
0709 12:18:58.423809 (+274999us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.487442 (+ 63633us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.487585 (+   143us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:58.487617 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.487683 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:58.769274 (+281591us) raft_consensus.cc:1685] finished
0709 12:18:58.769300 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.769456 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127619,"prepare.queue_time_us":357611,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":14771,"raft.run_cpu_time_us":518,"raft.run_wall_time_us":50650,"thread_start_us":361414,"threads_started":2,"wal-append.queue_time_us":595834}
I20260709 12:18:58.773029 25783 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:58.725648 26335 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:58.738317 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:58.777904 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:58.728734 26336 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
W20260709 12:18:58.780126 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.784441 25711 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:58.785929 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.788786 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.789613 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:58.788887 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.791617 25636 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:58.779409 26343 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:18:58.795918 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2240) took 1181 ms. Trace:
I20260709 12:18:58.796947 25783 rpcz_store.cc:276] 0709 12:18:57.592804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.772368 (+1179564us) service_pool.cc:224] Handling call
0709 12:18:58.772761 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.773906 (+  1145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.774160 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.800021 25712 raft_consensus.cc:2410] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:58.804458 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.801925 25708 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 12:18:58.795526 25787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.027s	sys 0.040s
W20260709 12:18:58.811822 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 86641.2us
I20260709 12:18:58.812381 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.813270 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2212) took 4540 ms. Trace:
I20260709 12:18:58.814939 25629 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
W20260709 12:18:58.816911 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.820443 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 540246us
I20260709 12:18:58.813925 25787 rpcz_store.cc:276] 0709 12:18:54.272469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.930284 (+3657815us) service_pool.cc:224] Handling call
0709 12:18:57.966977 (+ 36693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.967412 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:58.221797 (+254385us) log.cc:844] Serialized 14 byte log entry
0709 12:18:58.812569 (+590772us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:58.812685 (+   116us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:58.812811 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.812975 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.813176 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":378426,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"thread_start_us":772331,"threads_started":2}
I20260709 12:18:58.738729 29213 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:58.825892 25785 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.010s	sys 0.012s
I20260709 12:18:58.825742 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:58.830777 25785 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.011s	sys 0.012s
I20260709 12:18:58.836812 25711 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:58.839682 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.840612 25712 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:58.841868 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.843672 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.844503 25711 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:58.796663 29147 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:58.865515 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:58.822618 29251 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.869380 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:58.870370 25711 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:58.831027 29112 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.003s
I20260709 12:18:58.846398 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:58.829871 29209 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:58.824033 29022 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:58.868371 29221 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.004s
W20260709 12:18:58.882946 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 144788us
I20260709 12:18:58.897619 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:58.884544 25711 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:18:58.885056 29004 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:58.893517 29029 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.894708 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:58.896943 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:18:58.904960 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 90194.9us
I20260709 12:18:58.905743 25636 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:58.902364 29221 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.910285 25783 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:58.921097 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:58.922217 25783 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
W20260709 12:18:58.911486 29036 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.010s	sys 0.025s
W20260709 12:18:58.914876 25632 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.016s	sys 0.013s
W20260709 12:18:58.937825 25632 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.016s	sys 0.013s
I20260709 12:18:58.871901 25712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:18:58.938454 25712 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:58.938905 25712 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:58.937587 25708 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:18:58.952155 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:18:58.953881 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:58.957309 25704 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:18:58.941249 29036 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.010s	sys 0.027s
I20260709 12:18:58.918411 29191 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:58.824352 29035 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:58.909977 29112 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:58.973821 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 134975us
I20260709 12:18:59.007993 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.009178 29112 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.010829 29007 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:58.987772 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 507761us
W20260709 12:18:59.011996 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.999333 29209 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:58.999926 29249 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.015363 29249 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.017968 29249 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:58.912102 29251 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.025705 29251 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:59.000363 25631 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.003s	sys 0.012s
W20260709 12:18:59.026474 25631 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.003s	sys 0.013s
I20260709 12:18:59.010193 26307 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.020848 29004 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:59.040412 25786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.013s	sys 0.038s
I20260709 12:18:59.023373 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:18:59.040951 25786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.013s	sys 0.038s
W20260709 12:18:58.998174 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 85065.7us
I20260709 12:18:59.038659 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1770) took 1383 ms. Trace:
W20260709 12:18:59.039255 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 97110.3us
I20260709 12:18:59.039865 29163 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:59.021629 29007 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.045513 25630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.035s	sys 0.064s
I20260709 12:18:58.916644 29029 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.046852 25630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.036s	sys 0.064s
I20260709 12:18:59.046835 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.044183 25712 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:58.957654 29174 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.052917 29042 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.029s	sys 0.072s
W20260709 12:18:59.102140 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.114394 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69178.7us
W20260709 12:18:59.115540 29141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.021s	sys 0.069s
I20260709 12:18:59.123562 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.124547 25783 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
W20260709 12:18:59.126612 25784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.048s	sys 0.080s
W20260709 12:18:59.127264 25784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.049s	sys 0.080s
I20260709 12:18:59.041854 25631 rpcz_store.cc:276] 0709 12:18:57.644412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.649627 (+  5215us) service_pool.cc:224] Handling call
0709 12:18:57.650291 (+   664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.650805 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:58.011149 (+360344us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.026846 (+1015697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.026926 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:59.026982 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.027060 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.027199 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":538714,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":150665,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2543,"thread_start_us":1290153,"threads_started":3,"wal-append.queue_time_us":135424}
I20260709 12:18:59.138351 25712 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:18:59.120160 29042 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.030s	sys 0.072s
W20260709 12:18:59.127454 29141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.022s	sys 0.070s
I20260709 12:18:59.114286 29253 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8.
I20260709 12:18:59.144207 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:59.146160 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:59.156353 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:59.158663 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2218) took 2344 ms. Trace:
I20260709 12:18:59.160832 25785 rpcz_store.cc:276] 0709 12:18:56.814885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.225550 (+1410665us) service_pool.cc:224] Handling call
0709 12:18:58.295542 (+ 69992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.321064 (+ 25522us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:58.831361 (+510297us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.154882 (+323521us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.154968 (+    86us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:59.158271 (+  3303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.158376 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.158579 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25266,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":18192,"thread_start_us":765260,"threads_started":2}
W20260709 12:18:59.171976 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 148653us
I20260709 12:18:59.175221 25710 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:59.178824 26339 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:59.121718 29007 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.188248 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:59.184808 25637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.037s	sys 0.043s
W20260709 12:18:59.191838 25637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.037s	sys 0.043s
I20260709 12:18:59.192500 25785 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:18:59.194211 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 56739.3us
W20260709 12:18:59.197912 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 68955.8us
I20260709 12:18:59.156306 29249 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:18:59.140857 29174 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.208534 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 454871us
I20260709 12:18:59.192231 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:59.203475 25781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.013s	sys 0.005s
I20260709 12:18:59.186229 29042 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:59.210258 25781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.014s	sys 0.005s
I20260709 12:18:59.210708 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:59.213550 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.214193 25785 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:59.218151 25783 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:18:59.219383 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2235) took 1639 ms. Trace:
I20260709 12:18:59.225160 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:59.236570 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:59.251858 25778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.002s	sys 0.031s
W20260709 12:18:59.254979 25778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.002s	sys 0.031s
I20260709 12:18:59.222988 29258 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:59.212414 29163 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:59.222568 29159 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:59.211625 29042 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:59.227250 29203 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:59.205868 29141 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:18:59.264564 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.223500 29004 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:59.228721 29221 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:59.272697 25780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.008s	sys 0.029s
W20260709 12:18:59.273310 25780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.008s	sys 0.029s
I20260709 12:18:59.275592 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:59.273468 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.277228 25783 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
W20260709 12:18:59.271700 29216 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.029s
I20260709 12:18:59.260866 29258 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:59.284217 25787 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:59.268430 29042 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.265086 29163 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.271728 29043 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:59.289722 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2241) took 1692 ms. Trace:
I20260709 12:18:59.292126 25787 rpcz_store.cc:276] 0709 12:18:57.593186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.271290 (+1678104us) service_pool.cc:224] Handling call
0709 12:18:59.281457 (+ 10167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.284994 (+  3537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.285173 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:59.289628 25783 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:59.272428 29191 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:59.289083 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:59.293915 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2242) took 1700 ms. Trace:
W20260709 12:18:59.282197 29216 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.009s	sys 0.030s
I20260709 12:18:59.294296 25783 rpcz_store.cc:276] 0709 12:18:57.593609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.277710 (+1684101us) service_pool.cc:224] Handling call
0709 12:18:59.277904 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.293628 (+ 15724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.293856 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:59.285832 25709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.023s	sys 0.026s
W20260709 12:18:59.295599 25709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.023s	sys 0.026s
I20260709 12:18:59.273855 29004 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:18:59.297648 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 87042.7us
I20260709 12:18:59.308410 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2244) took 1714 ms. Trace:
I20260709 12:18:59.312393 25787 rpcz_store.cc:276] 0709 12:18:57.594480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.293266 (+1698786us) service_pool.cc:224] Handling call
0709 12:18:59.307871 (+ 14605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.308074 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.308103 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.308148 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:59.308183 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.308239 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.308352 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:59.286139 25536 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 5, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.298728 29004 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:59.288235 29251 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:59.287358 29258 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:59.289662 29029 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.289870 29163 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:59.327420 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 109386us
W20260709 12:18:59.311743 29040 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.568s	user 0.015s	sys 0.000s
I20260709 12:18:59.276401 29203 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.327916 28965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.017s	sys 0.023s
I20260709 12:18:59.344959 25783 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:59.333158 29251 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:59.330789 29245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.061s	sys 0.100s
I20260709 12:18:59.362003 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:59.326304 29259 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:59.327840 28941 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
W20260709 12:18:59.347369 28965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.017s	sys 0.024s
I20260709 12:18:59.375212 29209 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.379565 29209 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:59.380913 29260 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:18:59.374689 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 86548.5us
I20260709 12:18:59.382162 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 745) took 1183 ms. Trace:
W20260709 12:18:59.398192 25629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1776) took 1328 ms (client timeout 1505 ms). Trace:
I20260709 12:18:59.373744 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 561) took 1376 ms. Trace:
I20260709 12:18:59.403152 29209 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:59.404241 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2225) took 2581 ms. Trace:
I20260709 12:18:59.404811 25784 rpcz_store.cc:276] 0709 12:18:56.823325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.385922 (+1562597us) service_pool.cc:224] Handling call
0709 12:18:58.386284 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.445871 (+ 59587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.445917 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.503771 (+ 57854us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.127909 (+624138us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:59.127974 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.128080 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.403968 (+275888us) raft_consensus.cc:1685] finished
0709 12:18:59.403998 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.404162 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19452,"raft.queue_time_us":11334,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":26274,"thread_start_us":594262,"threads_started":1,"wal-append.queue_time_us":816735}
I20260709 12:18:59.400350 25783 rpcz_store.cc:276] 0709 12:18:57.997459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.314869 (+1317410us) service_pool.cc:224] Handling call
0709 12:18:59.325227 (+ 10358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.373341 (+ 48114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.373661 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:59.394939 25630 rpcz_store.cc:276] 0709 12:18:58.198622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.198891 (+   269us) service_pool.cc:224] Handling call
0709 12:18:58.219640 (+ 20749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.374802 (+155162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.374834 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.375459 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.052342 (+676883us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:59.052408 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.052522 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.381905 (+329383us) raft_consensus.cc:1685] finished
0709 12:18:59.381926 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.382084 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":676351,"threads_started":1,"wal-append.queue_time_us":876275}
I20260709 12:18:59.409005 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:18:59.398830 25629 rpcz_store.cc:269] 0709 12:18:58.055165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.055598 (+   433us) service_pool.cc:224] Handling call
0709 12:18:59.367532 (+1311934us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54003016d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:59.383799 (+ 16267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598802176}
I20260709 12:18:59.347029 29040 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.375403 29251 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.435736 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.437559 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.438814 29209 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:59.394795 29159 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.429904 29277 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:59.431139 29040 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:59.375906 29007 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:18:59.479737 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 135449us
I20260709 12:18:59.414788 29174 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:59.482296 29159 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.485618 25603 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:59.490621 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.491340 29265 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:18:59.492751 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2232) took 2602 ms. Trace:
W20260709 12:18:59.484050 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46627.5us
W20260709 12:18:59.538092 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 374348us
I20260709 12:18:59.558458 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:59.474689 28965 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:59.439831 29191 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:59.558903 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.558072 25778 rpcz_store.cc:276] 0709 12:18:56.890830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.548115 (+1657285us) service_pool.cc:224] Handling call
0709 12:18:58.548541 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.608276 (+ 59735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.608305 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.619176 (+ 10871us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.255735 (+636559us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:59.255797 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.255916 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.492345 (+236429us) raft_consensus.cc:1685] finished
0709 12:18:59.492391 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.492652 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17760,"raft.queue_time_us":60231,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":42093,"thread_start_us":531688,"threads_started":1,"wal-append.queue_time_us":797160}
W20260709 12:18:59.565873 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.482659 29249 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:59.475085 29022 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:59.423561 29245 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:59.479219 29203 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:18:59.579299 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.418843 29259 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.576560 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.575794 25707 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:18:59.588027 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 235196us
W20260709 12:18:59.580062 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 267180us
I20260709 12:18:59.567062 28965 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:59.595255 26307 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:18:59.597611 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.599294 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:59.599642 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:59.602202 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:59.603904 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:59.604324 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 496) took 1069 ms. Trace:
I20260709 12:18:59.604621 25634 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.604862 25709 rpcz_store.cc:276] 0709 12:18:58.531696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.531842 (+   146us) service_pool.cc:224] Handling call
0709 12:18:58.538081 (+  6239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.559444 (+ 21363us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:59.295956 (+736512us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.601024 (+305068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.601080 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:59.601125 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.601193 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.601332 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":990219,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"thread_start_us":948505,"threads_started":2}
W20260709 12:18:59.611876 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.615051 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:59.579182 29043 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:59.602059 29036 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:59.612161 25631 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:18:59.621093 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:59.597772 29249 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:59.595419 29221 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:59.622851 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64464.3us
I20260709 12:18:59.601109 29183 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:59.625341 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:18:59.646907 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 205972us
I20260709 12:18:59.611068 29174 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:59.640311 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2223) took 2817 ms. Trace:
I20260709 12:18:59.640139 29258 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.649187 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 456999us
I20260709 12:18:59.637801 25638 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:18:59.665853 25638 pending_rounds.cc:85] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 6
I20260709 12:18:59.665812 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:59.635562 29159 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:59.628723 28965 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.596004 29022 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.660020 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:18:59.663358 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1730) took 3297 ms. Trace:
I20260709 12:18:59.661499 25780 rpcz_store.cc:276] 0709 12:18:56.822732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.295653 (+1472921us) service_pool.cc:224] Handling call
0709 12:18:58.420237 (+124584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.524547 (+104310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.524595 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.538430 (+ 13835us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.273818 (+735388us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:59.273892 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.273995 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.639922 (+365927us) raft_consensus.cc:1685] finished
0709 12:18:59.639951 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.640181 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12769,"raft.queue_time_us":8927,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":7822,"thread_start_us":663065,"threads_started":1,"wal-append.queue_time_us":1002443}
I20260709 12:18:59.666500 25638 pending_rounds.cc:107] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:18:59.670316 25638 raft_consensus.cc:2889] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:59.649945 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 102219us
I20260709 12:18:59.622643 29245 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:59.626945 29249 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:59.673410 25634 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:18:59.673084 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:59.673313 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:59.679220 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:18:59.680145 25710 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:59.681468 29249 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:59.669821 25786 rpcz_store.cc:276] 0709 12:18:56.366236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.221997 (+1855761us) service_pool.cc:224] Handling call
0709 12:18:58.389868 (+167871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.417957 (+ 28089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.417986 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.445168 (+ 27182us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.041414 (+596246us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:59.129745 (+ 88331us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.130050 (+   305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.130152 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.662998 (+532846us) raft_consensus.cc:1685] finished
0709 12:18:59.663034 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.663265 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63483,"prepare.queue_time_us":88159,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":70,"raft.queue_time_us":25073,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":27086,"thread_start_us":526064,"threads_started":1,"wal-append.queue_time_us":758359}
I20260709 12:18:59.682948 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:18:59.683894 25712 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:59.684686 29191 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:59.685212 29266 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:18:59.685777 29022 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.668985 29159 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.684715 29271 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:18:59.663470 29036 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.687103 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2131) took 1471 ms. Trace:
I20260709 12:18:59.687644 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2234) took 2107 ms. Trace:
I20260709 12:18:59.688971 29036 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.689607 28965 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.689286 29043 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.683894 29004 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.690697 26309 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:59.691251 29203 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:59.694686 29204 raft_consensus.cc:2764] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:18:59.695994 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.696311 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:59.697283 29258 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.681887 29290 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:18:59.688064 25779 rpcz_store.cc:276] 0709 12:18:57.579909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.601030 (+1021121us) service_pool.cc:224] Handling call
0709 12:18:58.620822 (+ 19792us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.628687 (+  7865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.628728 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.765832 (+137104us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.212544 (+446712us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:59.212584 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.212664 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.687442 (+474778us) raft_consensus.cc:1685] finished
0709 12:18:59.687463 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.687584 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":446313,"threads_started":1,"wal-append.queue_time_us":672655}
W20260709 12:18:59.699575 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.668905 29270 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:59.694100 29203 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:59.715457 25707 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:59.687528 25637 rpcz_store.cc:276] 0709 12:18:58.222884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.223064 (+   180us) service_pool.cc:224] Handling call
0709 12:18:58.320322 (+ 97258us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:58.408796 (+ 88474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.408832 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:58.445463 (+ 36631us) log.cc:844] Serialized 52 byte log entry
0709 12:18:59.193495 (+748032us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:59.213148 (+ 19653us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.216847 (+  3699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.216955 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.686861 (+469906us) raft_consensus.cc:1685] finished
0709 12:18:59.686884 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.687041 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79453,"prepare.queue_time_us":19880,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":145937,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":16356,"thread_start_us":656933,"threads_started":1,"wal-append.queue_time_us":881568}
I20260709 12:18:59.685225 29191 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:59.727387 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82658.1us
I20260709 12:18:59.729691 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:59.669564 29042 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:18:59.734637 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 109395us
I20260709 12:18:59.741247 29007 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:59.742389 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46331.7us
I20260709 12:18:59.743284 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:59.744437 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2320) took 1065 ms. Trace:
I20260709 12:18:59.744599 29259 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.219893 25781 rpcz_store.cc:276] 0709 12:18:57.580369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.619943 (+1039574us) service_pool.cc:224] Handling call
0709 12:18:58.630806 (+ 10863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.631107 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:58.631482 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.218904 (+587422us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.218976 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:59.219045 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.219136 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.219308 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2820,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":6444,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":29702,"thread_start_us":580493,"threads_started":1}
I20260709 12:18:59.745332 29163 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.741014 26336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 625) took 2033 ms (client timeout 2105 ms). Trace:
W20260709 12:18:59.746837 26336 rpcz_store.cc:269] 0709 12:18:57.704972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.705259 (+   287us) service_pool.cc:224] Handling call
0709 12:18:59.737905 (+2032646us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54005a7154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:18:59.738046 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1966897920}
I20260709 12:18:59.747182 29294 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.748073 29090 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:59.748456 29203 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.741341 29261 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:59.730995 29191 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:59.749867 25676 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:59.747795 29294 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.686590 29159 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.760200 29159 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:59.760905 29159 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.754032 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.756857 29294 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.764097 29159 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.765468 29294 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:59.766918 29294 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:59.770401 29294 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.775086 29159 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:59.777163 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.783870 29159 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:59.793498 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 72113.7us
I20260709 12:18:59.794989 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:18:59.796236 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:18:59.733546 28941 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:59.797412 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53903us
I20260709 12:18:59.731128 29290 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.801973 29290 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:18:59.802873 29004 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:59.802994 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:59.721966 29022 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:18:59.744844 25705 rpcz_store.cc:276] 0709 12:18:58.678843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.770736 (+ 91893us) service_pool.cc:224] Handling call
0709 12:18:59.203978 (+433242us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b5800004b98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:18:59.211639 (+  7661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:59.259884 (+ 48245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.259932 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:59.280855 (+ 20923us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.598232 (+317377us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:59.598303 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.598386 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.744122 (+145736us) raft_consensus.cc:1685] finished
0709 12:18:59.744145 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.744370 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2675,"raft.queue_time_us":9230,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6734,"spinlock_wait_cycles":14804736,"thread_start_us":272910,"threads_started":1,"wal-append.queue_time_us":428232}
I20260709 12:18:59.811869 25710 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:18:59.748567 29090 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:18:59.751345 29259 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:59.824684 29022 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:59.751626 29261 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:18:59.749222 29203 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.818626 29288 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:59.842931 25637 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:59.822763 29090 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.816264 28941 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:59.822350 29290 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:59.742408 29007 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.850857 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 77853.2us
I20260709 12:18:59.822002 28965 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:59.833276 29022 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:18:59.846524 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43682.4us
I20260709 12:18:59.753610 29270 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:18:59.839452 29216 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.847389 29288 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.849888 29290 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.846050 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.841668 29261 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:59.918298 29213 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.045s	user 0.005s	sys 0.009s
I20260709 12:18:59.919217 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:59.919930 26339 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:59.855556 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:18:59.920687 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 65212.4us
I20260709 12:18:59.859958 29292 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:59.921373 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:18:59.922204 29292 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:59.872411 29264 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.623s	user 0.001s	sys 0.004s
I20260709 12:18:59.922784 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:59.923053 29299 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:59.872934 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.874249 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.875448 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.869876 29288 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:59.924124 26341 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:18:59.861601 29296 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:18:59.924285 26342 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:18:59.925201 29296 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:59.925376 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.876471 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.925750 29040 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:59.926023 26342 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:18:59.875154 25536 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 8, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 8 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:18:59.926698 29290 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:59.927109 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:18:59.926272 29040 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.927893 29040 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:59.927791 25536 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.925103 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2301) took 1753 ms. Trace:
I20260709 12:18:59.929405 25536 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.926337 26340 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:59.876489 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.930987 26335 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:59.931784 29163 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:18:59.931843 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:59.929967 25713 rpcz_store.cc:276] 0709 12:18:58.202246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.207725 (+  5479us) service_pool.cc:224] Handling call
0709 12:18:58.227822 (+ 20097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.293086 (+ 65264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.293118 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.420366 (+127248us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.918772 (+498406us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:58.918828 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.918926 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.871622 (+952696us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:59.924742 (+ 53120us) raft_consensus.cc:1685] finished
0709 12:18:59.924765 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.925031 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45387,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":6910720,"thread_start_us":422818,"threads_started":1,"wal-append.queue_time_us":738464}
I20260709 12:18:59.932523 26341 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:59.860071 29251 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.874325 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:18:59.935111 29289 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:18:59.925639 29296 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:18:59.935683 29251 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:59.892036 25603 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:18:59.935982 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71428.4us
I20260709 12:18:59.936089 29296 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:18:59.937074 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79934us
I20260709 12:18:59.937819 29296 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:59.937958 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:18:59.908598 25533 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 303) took 30862 ms (client timeout 15000 ms). Trace:
W20260709 12:18:59.851217 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121619us
I20260709 12:18:59.869621 29216 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.938423 29283 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:59.931100 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.939262 29283 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:18:59.939160 28941 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.841979 29259 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:18:59.940474 29283 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.940799 29259 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.930094 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.815551 29245 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:18:59.942648 29253 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:59.844300 29258 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:18:59.876149 29035 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.863546 29022 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.866398 29293 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:59.934978 29183 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:59.939525 26340 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:18:59.899919 28983 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:59.924209 26343 raft_consensus.cc:2410] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:59.879969 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:18:59.881875 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.918962 29213 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:18:59.946408 25536 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.937898 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:18:59.932364 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:18:59.970186 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:59.969941 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:59.979890 25787 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.923465 29042 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:18:59.970582 25636 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:59.859337 29004 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:18:59.961467 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.980504 25632 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:59.972676 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.989530 25779 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:59.989764 25781 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:18:59.968603 26342 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:18:59.960114 29209 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.006s
I20260709 12:18:59.976637 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:59.967867 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:18:59.998545 25633 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:18:59.977026 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:59.970445 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:18:59.985236 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:18:59.985359 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:00.001135 25786 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:18:59.922766 29292 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:00.002240 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64434.2us
I20260709 12:18:59.977417 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:18:59.986752 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.969892 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:00.003566 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.004256 25778 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:00.004467 25709 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:18:59.998617 25783 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:59.967697 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:18:59.997023 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.006897 25704 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:00.006280 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.012086 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:00.012449 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 216278us
I20260709 12:19:00.013473 25785 raft_consensus.cc:2368] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:00.005482 25783 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:00.015131 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 119625us
I20260709 12:19:00.012338 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.016517 25636 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:00.018231 25713 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:18:59.968864 29270 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:00.020385 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.020893 25785 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:00.025007 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.025812 25705 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:59.995805 29258 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:19:00.008031 29209 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:00.032111 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:18:59.959743 29299 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:19:00.042266 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:00.047024 25779 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.042266 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:00.037349 25783 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.042630 25536 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:00.043355 25712 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:19:00.048763 25712 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:00.014145 28965 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:00.008649 29292 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:00.049494 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.044571 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.051618 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.046160 25750 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:00.052004 25637 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:18:59.996588 29261 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.056521 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.052582 25705 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:00.001474 29253 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:19:00.061765 25783 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:00.062683 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:00.059689 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.029247 29029 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 12:18:59.947736 25533 rpcz_store.cc:269] 0709 12:18:29.046091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.046377 (+   286us) service_pool.cc:224] Handling call
0709 12:18:59.908289 (+30861912us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:53.948747 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:53.973344 (+ 24597us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:53.973417 (+    73us) write_op.cc:435] Acquired schema lock
0709 12:18:53.973441 (+    24us) tablet.cc:662] Decoding operations
0709 12:18:54.085160 (+111719us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:54.085236 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:54.085272 (+    36us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:18:54.191730 (+106458us) tablet.cc:701] Row locks acquired
0709 12:18:54.191751 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:18:54.191909 (+   158us) write_op.cc:270] Start()
0709 12:18:54.192029 (+   120us) write_op.cc:276] Timestamp: P: 1783599534191866 usec, L: 0
0709 12:18:54.192050 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:18:54.250767 (+ 58717us) log.cc:844] Serialized 5169 byte log entry
0709 12:18:54.590878 (+340111us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:18:55.196099 (+605221us) op_driver.cc:464] REPLICATION: finished
0709 12:18:55.434862 (+238763us) write_op.cc:301] APPLY: starting
0709 12:18:55.436734 (+  1872us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:55.505501 (+ 68767us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:55.505524 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:55.507294 (+  1770us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:55.507680 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:18:55.507756 (+    76us) write_op.cc:312] APPLY: finished
0709 12:18:55.508246 (+   490us) log.cc:844] Serialized 95 byte log entry
0709 12:18:55.508480 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:55.508734 (+   254us) write_op.cc:454] Released schema lock
0709 12:18:55.508810 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:55.509277 (+   467us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:55.509302 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1169912,"reported_tablets":32,"spinlock_wait_cycles":19328,"child_traces":[["op",{"apply.queue_time_us":207127,"apply.run_cpu_time_us":4591,"apply.run_wall_time_us":80666,"num_ops":13,"prepare.queue_time_us":510438,"prepare.run_cpu_time_us":26121,"prepare.run_wall_time_us":398587,"raft.queue_time_us":361633,"raft.run_cpu_time_us":21260,"raft.run_wall_time_us":229605,"replication_time_us":980533,"spinlock_wait_cycles":10268416,"thread_start_us":811976,"threads_started":4,"wal-append.queue_time_us":241770,"wal-append.run_cpu_time_us":16309,"wal-append.run_wall_time_us":2091726}]]}
I20260709 12:19:00.068624 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.065527 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:00.073074 26341 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:00.072604 25713 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:00.072932 25709 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:00.094717 25787 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
W20260709 12:19:00.119067 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 95921us
W20260709 12:19:00.122112 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60887.1us
I20260709 12:19:00.048616 29043 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.132833 25779 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:00.047654 29270 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:00.134119 25710 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:19:00.135556 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.136232 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.123651 25705 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:19:00.124132 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:00.124291 25634 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:00.122287 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58602.5us
I20260709 12:19:00.063540 25712 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.141093 25704 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:00.144160 25706 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:00.073331 26339 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:19:00.071643 29006 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:00.127722 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.131315 29007 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:00.071974 29107 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:00.146642 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:00.219820 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 74112.6us
W20260709 12:19:00.221403 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2322) took 1468 ms (client timeout 1931 ms). Trace:
W20260709 12:19:00.221824 25706 rpcz_store.cc:269] 0709 12:18:58.752936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.762737 (+  9801us) service_pool.cc:224] Handling call
0709 12:18:59.874155 (+1111418us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5800004b98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:19:00.221238 (+347083us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400073fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:00.221345 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323948672}
I20260709 12:19:00.067188 29216 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:00.029731 29300 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 12:19:00.067906 29203 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:00.229099 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:00.224298 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:00.224694 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 80060.5us
I20260709 12:19:00.225270 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.225190 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:00.160763 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:00.069661 29213 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:00.063413 29281 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.171s
I20260709 12:19:00.252713 29253 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:00.253168 29253 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:00.255273 29253 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:00.256444 29249 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.003s
I20260709 12:19:00.256682 29253 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:00.257010 29249 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.147063 29006 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:00.217985 28965 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:00.259372 29022 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.260311 28965 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.120546 29040 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.147689 29042 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 12:19:00.224802 29107 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.252341 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:00.263814 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:00.272912 26338 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:00.264770 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.245627 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 113231us
I20260709 12:19:00.067909 29209 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:00.064733 29304 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:00.186496 25704 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:00.230726 29029 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:19:00.230899 26341 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:00.231017 29007 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:19:00.231608 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 70948.3us
W20260709 12:19:00.132337 29260 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.004s
I20260709 12:19:00.073923 25536 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 5 to 7, term changed from 6 to 7, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to <none>, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 7 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:00.243196 25783 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:00.281839 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:00.244499 25706 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:00.245266 25713 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.277146 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:00.283749 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:00.150782 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.144919 29112 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:00.229420 29183 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:00.289335 25705 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:19:00.294730 25536 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:00.266412 29297 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:00.250685 29245 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:00.265869 29107 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.218461 29043 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.290692 29042 raft_consensus.cc:2764] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:19:00.263583 29040 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:00.261739 29022 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:00.125622 29294 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.157s
I20260709 12:19:00.299741 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:00.327380 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:00.330720 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.279093 29007 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:00.335660 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.323930 29297 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.245587 29303 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:00.328518 25536 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 7 to 10, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 10 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:00.295004 29029 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:00.379354 25707 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:00.294018 29213 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:00.299685 29293 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:00.278916 29141 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:19:00.348052 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:00.388159 26339 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:00.263491 29036 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:19:00.349788 25705 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:00.358827 29159 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:00.362762 29270 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.250197 29291 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.308s
I20260709 12:19:00.370153 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:00.323761 29245 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:00.375562 25536 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 5, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:00.288563 29209 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:00.406821 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79567.4us
I20260709 12:19:00.407707 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:00.383679 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 318226us
I20260709 12:19:00.297263 29251 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:19:00.379285 29029 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:00.391305 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:00.294790 29191 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.368556 29007 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.381347 29297 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:00.395175 29042 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:00.408762 25705 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:00.381620 29163 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:19:00.411216 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.411020 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:00.384670 29304 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.413267 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:00.385557 26338 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:00.418087 26336 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:00.418975 25536 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:00.406481 29303 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:00.423234 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.403518 29022 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:00.432519 25536 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:00.411614 29203 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:00.409559 29159 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.436810 25536 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 5, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 5 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:00.438975 25536 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:00.415231 29249 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:00.417799 29298 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:00.442353 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:00.443745 25783 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.440327 25536 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 5, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 5 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:00.445250 25709 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:19:00.448904 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.421078 29042 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:00.424436 29163 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.485498 25783 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:00.468606 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.472795 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:00.474014 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:00.429142 29296 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:00.438313 29159 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:00.424521 29303 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:00.486912 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 151336us
I20260709 12:19:00.478753 25536 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:00.489775 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:00.441334 29216 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:00.440104 29203 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.513700 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.444155 29261 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:00.517081 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.479106 29029 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.469331 29298 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.522257 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:00.525835 29294 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:00.526232 25676 leader_election.cc:400] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:00.526794 25676 leader_election.cc:403] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:00.491797 29163 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:19:00.531070 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62599.8us
I20260709 12:19:00.533279 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.534126 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.439316 29283 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:19:00.538892 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:00.512243 29022 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:00.491405 29296 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:00.554392 25632 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.000s
I20260709 12:19:00.558646 25632 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:00.526356 29294 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.543975 29321 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:19:00.472815 29315 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:19:00.540967 29040 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:19:00.529891 29298 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:00.547577 29183 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:00.548398 29029 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.548455 29107 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.519706 29203 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.518811 29245 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.550804 29322 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:19:00.551347 29035 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.000s
I20260709 12:19:00.524179 28983 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:19:00.520093 29213 raft_consensus.cc:2764] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:19:00.557135 25712 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:19:00.514539 29303 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.577306 29294 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:00.429345 29304 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:00.560253 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49026.2us
W20260709 12:19:00.594674 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 54411.3us
W20260709 12:19:00.564523 29090 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.000s
W20260709 12:19:00.602784 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 44787.7us
I20260709 12:19:00.617246 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:00.541800 29043 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:00.560391 29296 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:00.563788 29293 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:00.529593 29301 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:00.560443 29004 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:00.607095 29112 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:00.615703 25635 raft_consensus.cc:2410] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:19:00.614048 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 80061.9us
I20260709 12:19:00.602890 29322 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:00.632375 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:00.609123 29221 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.605974 29022 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:00.643381 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.644385 25713 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
W20260709 12:19:00.633422 25708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.009s	sys 0.014s
W20260709 12:19:00.655527 25708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.009s	sys 0.014s
W20260709 12:19:00.659432 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.620573 29301 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.618945 29183 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:00.668545 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51400us
I20260709 12:19:00.622757 29004 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:00.635574 29043 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:00.673378 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.636205 29112 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.674170 25713 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:00.634865 29293 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:00.632720 29216 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:00.599851 29042 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.682168 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.620584 29090 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.660703 29258 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.666648 29213 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:00.672842 29301 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.661746 29209 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:00.702036 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.687626 29043 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:00.731103 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.673568 29191 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:00.735379 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.680917 29253 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:00.685725 29313 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:19:00.659516 29221 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.738759 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.681821 29112 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:00.740486 25712 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:00.741510 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:00.699435 29040 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.746198 25630 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:00.736060 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.754794 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.736975 28941 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.756134 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.740928 28983 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:00.759687 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.764698 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:00.766284 25713 raft_consensus.cc:2376] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6. Re-sending same reply.
I20260709 12:19:00.767736 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.759634 25638 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:19:00.770781 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.771569 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.770015 25707 raft_consensus.cc:2376] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:00.774909 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.751579 29245 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:00.775245 25633 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:00.775518 25631 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:00.778061 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:00.732131 29249 consensus_queue.cc:1175] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "955233c139344131a847044f17f8d348" responder_term: 9 status { last_received { term: 9 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 9 index: 6 } }
I20260709 12:19:00.782706 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:00.783442 29159 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:00.784389 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:00.784267 29293 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.785204 29040 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.787212 29216 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.787981 29040 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:00.789179 29107 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:00.780589 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.738376 29258 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.794732 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.797851 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.798295 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:00.795127 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.798892 26338 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.799364 25707 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:00.802872 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:00.803535 25631 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:00.805665 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 73172.7us
I20260709 12:19:00.806039 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:00.808115 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:00.810575 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.812559 28983 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.741555 29022 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.816746 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:00.796150 29331 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:00.821652 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:00.798749 29299 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:00.822605 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:00.767410 29253 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.827953 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.834292 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:00.773746 28941 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.790194 29183 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.799721 29322 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.878206 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:00.845876 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:00.847849 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:00.850458 26338 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:00.847476 25785 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:19:00.850816 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:00.889178 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:00.925767 26344 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:00.850660 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.899134 25631 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:00.936131 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.936975 25631 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:00.934346 25633 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:19:00.934096 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:00.875617 25783 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:00.940960 25707 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:00.778462 29004 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.949178 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:00.842017 29339 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:19:00.863211 29327 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:19:00.950387 25637 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:19:00.872540 29288 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.948s	user 0.009s	sys 0.001s
W20260709 12:19:00.926448 29290 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.999s	user 0.012s	sys 0.000s
I20260709 12:19:00.838367 29107 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:00.842840 29291 raft_consensus.cc:2764] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:19:00.854828 29331 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:00.857769 29221 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:00.963614 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:00.968485 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.968662 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 134446us
I20260709 12:19:00.940718 29163 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.845271 29340 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:00.973055 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:00.956118 29339 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.980110 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:00.988867 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:00.989050 29330 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:19:00.990600 25707 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:00.990931 29329 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:00.975440 29299 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:00.997953 29326 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:19:00.874454 29006 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.973376 29290 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:00.999068 29339 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.972872 29107 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:00.843667 29043 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:01.013525 29292 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.881s	user 0.008s	sys 0.003s
I20260709 12:19:01.020570 29253 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.031968 29291 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:00.973246 29288 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:01.021240 29297 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.002s
I20260709 12:19:00.975497 29331 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.016621 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:01.029328 29298 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:01.064757 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 86075.7us
I20260709 12:19:01.086119 25779 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:00.998342 29337 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:01.089365 25787 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:01.057003 29221 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.088059 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:01.084507 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 241365us
W20260709 12:19:01.072340 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 183272us
I20260709 12:19:01.073428 29348 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:01.101513 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:01.102754 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:01.096084 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2385) took 1038 ms. Trace:
I20260709 12:19:01.105465 25710 raft_consensus.cc:2368] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6.
I20260709 12:19:01.109704 25633 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:19:01.112629 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:01.089159 29331 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:01.064777 29296 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.131748 25713 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:01.122509 29354 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:01.123404 29333 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:01.124480 29298 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.104599 29253 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:19:01.104851 29348 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.125818 29036 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:19:01.126456 29159 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:01.127918 29288 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.129477 29107 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:01.130781 25707 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:01.131685 25711 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:01.107438 29291 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:01.131748 29203 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 12:19:01.132501 25709 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:19:01.132601 25704 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
W20260709 12:19:01.132890 29007 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.706s	user 0.005s	sys 0.006s
I20260709 12:19:01.133188 29328 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:19:01.130218 29261 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
W20260709 12:19:01.138027 29294 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.001s
I20260709 12:19:01.108057 29221 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.139300 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2396) took 1019 ms. Trace:
I20260709 12:19:01.143250 29337 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:01.098474 29006 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:01.153869 25787 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:19:01.130326 29029 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.004s
I20260709 12:19:01.089038 29297 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.073311 29299 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:01.180547 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.180716 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:01.171363 25705 rpcz_store.cc:276] 0709 12:19:00.119605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.422719 (+303114us) service_pool.cc:224] Handling call
0709 12:19:00.423176 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.436940 (+ 13764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.436979 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.437496 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:19:00.603165 (+165669us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:00.603209 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.603275 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.139070 (+535795us) raft_consensus.cc:1685] finished
0709 12:19:01.139096 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.139243 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6444,"raft.queue_time_us":6665,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":165273,"threads_started":1,"wal-append.queue_time_us":233267}
I20260709 12:19:01.161039 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:01.167372 29036 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:01.167227 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.087428 29292 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.100246 29270 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:19:01.172115 26337 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:01.174898 29328 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:01.175832 29261 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:01.108202 29330 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:01.155725 29333 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:01.156000 29354 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.185052 25630 raft_consensus.cc:2410] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:19:01.159226 29348 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:01.202013 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.203477 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.173305 29337 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.235211 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.235715 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.238095 25713 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:01.185475 28941 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.236330 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:01.197407 29036 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:01.232306 29321 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.000s
I20260709 12:19:01.173564 29007 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.173791 29203 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.178851 29294 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.227111 29328 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:01.230263 29261 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:01.236012 29213 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:01.256018 29191 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:01.239359 29337 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:01.237519 29270 consensus_queue.cc:1243] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:01.264782 29203 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.265121 29112 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.266069 29355 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:19:01.267060 29362 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.251380 29321 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:01.239579 29354 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:01.242552 29163 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:01.245699 29159 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:01.269109 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:01.279919 29346 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:19:01.279644 29261 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.271517 29325 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.176s
I20260709 12:19:01.282752 29333 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.285013 29294 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.285351 29036 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.286392 29299 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.241060 29301 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.298344 29159 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:01.304600 29351 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:19:01.273996 29290 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.316910 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.339175 25713 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:01.325541 25711 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:01.317584 29029 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:01.361666 29296 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.361997 29029 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:01.363234 29029 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.364617 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:01.366315 29183 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:01.355708 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 875) took 1476 ms. Trace:
I20260709 12:19:01.366952 26336 rpcz_store.cc:276] 0709 12:18:59.879104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.932064 (+ 52960us) service_pool.cc:224] Handling call
0709 12:19:01.355521 (+1423457us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54006df454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:01.355635 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007263616}
I20260709 12:19:01.295500 29355 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:01.332024 29358 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:19:01.347028 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:01.384950 25713 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:01.353649 25635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1792) took 1752 ms (client timeout 1705 ms). Trace:
I20260709 12:19:01.355672 29007 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:01.356631 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:01.357604 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.359185 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.358670 29328 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.360675 25629 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:19:01.408850 25629 pending_rounds.cc:85] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 5
I20260709 12:19:01.409170 25629 pending_rounds.cc:107] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:19:01.409412 25629 raft_consensus.cc:2889] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:01.361119 29301 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.338310 29221 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.364521 29249 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:19:01.397673 26307 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.756s (SENT) [suppressed 27 similar messages]
I20260709 12:19:01.370378 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:01.329288 29112 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:01.327785 29036 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:01.386720 25635 rpcz_store.cc:269] 0709 12:18:59.601518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.601973 (+   455us) service_pool.cc:224] Handling call
0709 12:19:01.353374 (+1751401us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003011d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:01.353562 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328382208}
W20260709 12:19:01.369100 29303 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.000s
I20260709 12:19:01.298463 29163 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.419134 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.424002 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.312691 29352 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:19:01.270742 29291 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.369051 29183 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:19:01.444491 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74173.5us
I20260709 12:19:01.424805 25781 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:01.427597 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:01.445915 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 284920us
I20260709 12:19:01.422847 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.447100 25779 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:01.431200 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.448112 25786 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:01.434477 25780 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:01.364552 29029 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.440009 25785 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:19:01.376722 29319 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.458477 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.451265 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:01.450098 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:01.453740 25713 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
W20260709 12:19:01.466464 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.429796 29321 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:01.309473 29362 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.479211 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:01.484001 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:01.484119 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:01.485088 26338 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:01.470577 29007 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:01.480796 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.491711 25713 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:19:01.463414 29301 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:01.473959 29159 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.432729 29303 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.474687 29328 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:01.449323 29163 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:01.506225 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 108584us
W20260709 12:19:01.494690 29245 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.000s
I20260709 12:19:01.510025 25632 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:01.507992 26336 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:01.487517 29292 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.510207 25705 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:01.529870 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:01.530890 26341 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:01.535755 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:01.538132 26337 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:01.510330 29297 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.483042 29321 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.547605 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:01.548015 29303 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.553093 29350 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:01.518556 29174 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:01.565096 29174 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.469091 29319 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.567521 29174 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.497085 29007 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.568656 29319 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.569849 29007 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.560312 26341 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:01.559082 26342 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:01.548934 29365 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:19:01.570206 29350 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:01.570760 25779 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.577170 29163 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.496196 29355 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.589416 25786 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:19:01.611893 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47234.5us
I20260709 12:19:01.584905 25779 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
W20260709 12:19:01.596935 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.613472 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:01.599833 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 72628.9us
I20260709 12:19:01.614184 26341 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:01.544999 29304 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:01.620217 25710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2392) took 1555 ms (client timeout 2050 ms). Trace:
W20260709 12:19:01.622190 25710 rpcz_store.cc:269] 0709 12:19:00.064292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.135097 (+ 70805us) service_pool.cc:224] Handling call
0709 12:19:01.619971 (+1484874us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b5400147654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:01.620143 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011985280}
I20260709 12:19:01.575327 29350 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.630141 29007 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.635044 29281 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:01.649490 29191 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:01.684597 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.688865 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.689687 25786 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:01.689301 29375 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 12:19:01.690858 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.691550 25782 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:01.104001 25708 rpcz_store.cc:276] 0709 12:19:00.057471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.057590 (+   119us) service_pool.cc:224] Handling call
0709 12:19:00.093734 (+ 36144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.093938 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:19:00.659922 (+565984us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.095617 (+435695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.095691 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:01.095752 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.095845 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.096008 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":107677,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":454,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":8623,"thread_start_us":877428,"threads_started":2}
I20260709 12:19:01.679248 29281 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.697562 28983 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:01.697851 29281 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.698081 28983 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:01.699208 28983 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.699992 29348 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.700903 28983 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.703056 25779 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:01.709766 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.710541 25779 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.716372 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.664459 29007 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.742964 29321 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.743763 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.744851 29362 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:01.745720 29377 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:19:01.747067 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.747808 25782 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:01.748724 25708 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:19:01.753922 25781 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:01.761384 25779 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:01.752555 29331 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
I20260709 12:19:01.764540 29331 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.765230 25707 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:01.767063 25706 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:01.767273 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.768054 25782 raft_consensus.cc:2410] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:19:01.757956 29339 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.005s
W20260709 12:19:01.757704 29317 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.383s	user 0.006s	sys 0.008s
I20260709 12:19:01.751236 29325 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.775416 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.778879 25779 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:19:01.770959 29373 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:19:01.785262 26342 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:19:01.787943 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:01.788621 26341 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:01.791221 25782 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:19:01.792562 25707 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:01.794579 29007 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.773659 29339 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.795943 29258 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.796952 29339 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:01.798660 29331 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:01.799312 29337 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 12:19:01.799813 29337 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.802685 25635 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:01.763454 29354 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:01.804126 29006 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 12:19:01.804718 25638 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:01.805028 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:01.805691 25635 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:01.805847 29354 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:01.805783 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2145) took 1049 ms. Trace:
I20260709 12:19:01.806432 25638 rpcz_store.cc:276] 0709 12:19:00.756011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.771316 (+ 15305us) service_pool.cc:224] Handling call
0709 12:19:01.805615 (+1034299us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003007d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:01.805727 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2374698368}
I20260709 12:19:01.807542 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:01.807760 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2135) took 1110 ms. Trace:
I20260709 12:19:01.808082 25629 rpcz_store.cc:276] 0709 12:19:00.696908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.697034 (+   126us) service_pool.cc:224] Handling call
0709 12:19:01.360181 (+663147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.420414 (+ 60233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.807584 (+387170us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54003011d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:01.807709 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437438336}
I20260709 12:19:01.808600 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:01.809234 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:01.809311 25638 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 4.
I20260709 12:19:01.810038 25784 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:01.810137 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:01.791947 29191 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.814101 25635 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:01.814785 29042 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:01.815255 29341 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.969s	user 0.000s	sys 0.004s
I20260709 12:19:01.816922 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:01.818470 29378 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:19:01.820057 29348 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.782734 29325 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.821674 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:01.826918 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1802) took 1547 ms. Trace:
I20260709 12:19:01.827718 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:01.829324 26307 leader_election.cc:336] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 2.156s (SENT)
I20260709 12:19:01.832788 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.833441 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:01.833565 25711 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:01.834270 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:01.836850 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:01.837770 26341 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:01.838395 29354 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:01.839859 29337 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:01.840045 29325 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.797989 29350 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.842607 29350 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.847606 25779 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:01.827499 25636 rpcz_store.cc:276] 0709 12:19:00.279876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.280037 (+   161us) service_pool.cc:224] Handling call
0709 12:19:00.293933 (+ 13896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.300630 (+  6697us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:00.347590 (+ 46960us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.808906 (+461316us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.809336 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:19:00.809459 (+   123us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:00.809488 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.809555 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.814683 (+1005128us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:01.821399 (+  6716us) raft_consensus.cc:1685] finished
0709 12:19:01.826692 (+  5293us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.826842 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":945,"prepare.queue_time_us":63990,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":7946,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":22284,"spinlock_wait_cycles":95374592,"thread_start_us":430458,"threads_started":1,"wal-append.queue_time_us":467836}
I20260709 12:19:01.851014 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:01.852169 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:01.862934 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.863680 25711 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:01.876044 26341 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:01.891453 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:01.892371 25636 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:01.894141 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:01.895049 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.867197 29325 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.898854 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.904508 25784 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.899570 29006 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.900553 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.905436 25629 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:01.906409 25781 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:01.905808 29261 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:01.907903 25778 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 12:19:01.910727 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.913113 25786 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:19:01.908630 29006 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.915884 29006 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.918926 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.923313 29333 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:01.921582 25638 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
W20260709 12:19:01.926954 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.929396 25778 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:01.931326 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2133) took 1249 ms. Trace:
I20260709 12:19:01.918154 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:01.931475 26341 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:01.932561 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:01.926169 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:01.931752 25632 rpcz_store.cc:276] 0709 12:19:00.681569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.681893 (+   324us) service_pool.cc:224] Handling call
0709 12:19:01.931177 (+1249284us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54002abcd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:01.931272 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865455360}
I20260709 12:19:01.934861 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.935559 25784 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:01.936530 26342 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:01.936276 29328 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:01.939272 29029 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:01.939561 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:01.939739 26339 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:19:01.940375 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.941393 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.938617 29006 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:19:01.939087 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:01.942922 26339 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:01.944278 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2138) took 1243 ms. Trace:
I20260709 12:19:01.944561 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.945288 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:01.945470 25781 raft_consensus.cc:2368] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6.
I20260709 12:19:01.944701 25637 rpcz_store.cc:276] 0709 12:19:00.701332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.701471 (+   139us) service_pool.cc:224] Handling call
0709 12:19:00.739578 (+ 38107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.967465 (+227887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.944059 (+976594us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54001aead4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:01.944218 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483444608}
I20260709 12:19:01.946023 26341 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:01.949306 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:01.949620 29029 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:01.950151 29029 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:01.951300 29090 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:01.951081 29029 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.951742 29007 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:01.952086 29354 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:01.952687 29007 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:01.952801 29029 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:01.953107 29007 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:01.931735 29325 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.951067 29328 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:01.956135 25637 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:01.956527 25637 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:01.957023 29325 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:01.957614 29325 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:01.957980 29325 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:01.950237 26342 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:01.954161 29007 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.960960 29007 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:01.963704 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:01.964378 25785 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.966535 29006 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:01.967716 29006 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:01.966842 25750 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:01.969301 25632 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:19:01.971323 25632 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:19:01.957216 25637 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:01.973094 25637 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:01.976400 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:01.983659 25636 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:01.984037 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:01.984089 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:01.988514 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:01.992286 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:01.998874 26341 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:01.986622 29337 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:01.990597 29107 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:01.990329 29315 raft_consensus.cc:2764] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:19:01.997403 29350 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:19:02.005646 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.998786 28983 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:02.000046 29362 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:01.998986 25785 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:01.947463 29261 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 12:19:02.019726 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 68856.2us
I20260709 12:19:02.001765 29337 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:02.019728 29388 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:19:01.998459 29339 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:19:02.006130 29107 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:02.021301 29362 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.041177 29393 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.040977 29107 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.025497 29337 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:02.022246 29350 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:02.063432 29350 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:01.999607 29325 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.000026 29006 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.068874 25637 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:02.071033 25633 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:19:02.072535 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2146) took 1306 ms. Trace:
I20260709 12:19:02.073266 25630 rpcz_store.cc:276] 0709 12:19:00.766210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.987253 (+221043us) service_pool.cc:224] Handling call
0709 12:19:02.068397 (+1081144us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b540023fd54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:02.072441 (+  4044us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227089792}
I20260709 12:19:02.055415 29362 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:02.084306 29390 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:19:02.088646 29325 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:02.089931 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.075280 29258 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:19:02.092242 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:02.096611 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.097815 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:02.094940 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:02.099745 25784 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:02.101998 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.058092 29393 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.115134 25784 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:02.116403 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.118937 26307 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:02.112913 25601 connection.cc:256] RPC call timeout handler was delayed by -2.4333596070000567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 12:19:02.120617 25601 connection.cc:256] RPC call timeout handler was delayed by -2.3999752469999294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21050000000000002s for a response
I20260709 12:19:02.107118 29022 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:02.108546 29315 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:02.151678 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.109872 29348 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:02.151075 29298 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:02.143491 29213 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:02.160764 29394 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:02.181355 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2398) took 1198 ms. Trace:
I20260709 12:19:02.182631 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.188426 25783 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:02.187959 29394 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.189456 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.192235 26344 raft_consensus.cc:2410] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:19:02.192414 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:02.193858 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:02.194355 26339 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:02.194520 26335 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:02.187991 25706 rpcz_store.cc:276] 0709 12:19:00.983114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.983281 (+   167us) service_pool.cc:224] Handling call
0709 12:19:00.989932 (+  6651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.767512 (+777580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.181017 (+413505us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54002256d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:02.181264 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786817280}
I20260709 12:19:02.197389 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:02.200208 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:02.200798 25783 raft_consensus.cc:2376] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:02.202672 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:02.203976 29221 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:02.204597 29221 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:02.205529 29221 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.206249 29006 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:02.201718 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.209884 25783 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:02.226938 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.227900 25783 raft_consensus.cc:2376] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:02.227404 29393 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:02.228492 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2339) took 1026 ms. Trace:
I20260709 12:19:02.228771 25677 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.230136 25677 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:02.232836 29288 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.232952 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:02.233665 25638 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:02.233559 29288 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.232306 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.235791 29288 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:02.236346 29288 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:19:02.236452 25783 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:02.236847 29288 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:02.237785 26307 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.239440 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.241240 29321 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:02.241997 25783 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:02.237784 29288 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:02.217456 29387 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.211s	user 0.002s	sys 0.000s
I20260709 12:19:02.229038 25709 rpcz_store.cc:276] 0709 12:19:01.201910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.205390 (+  3480us) service_pool.cc:224] Handling call
0709 12:19:01.227286 (+ 21896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.227941 (+   655us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:19:01.424916 (+196975us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.545110 (+120194us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.545614 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.546630 (+  1016us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:01.546861 (+   231us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.227894 (+681033us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b18001fedc0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:02.228103 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.228178 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.228305 (+   127us) raft_consensus.cc:1685] finished
0709 12:19:02.228321 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.228438 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10608,"prepare.queue_time_us":394034,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":681488,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":7408,"spinlock_wait_cycles":81482624,"thread_start_us":257768,"threads_started":2,"wal-append.queue_time_us":199899}
I20260709 12:19:02.247923 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.251183 25704 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:02.187115 29298 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:02.249078 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.255807 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.218779 29394 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.257555 26307 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:02.256592 25783 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:02.258424 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:02.265411 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 108335us
W20260709 12:19:02.266180 25601 connection.cc:256] RPC call timeout handler was delayed by -1.3597680910002055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 12:19:02.266389 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.267151 25704 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:02.267473 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.265894 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.269783 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.270910 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.271961 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.272989 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:02.273663 25638 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:02.274235 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:02.281598 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.285789 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.288347 29090 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:02.289129 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.290542 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.291047 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.291249 25710 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:02.292542 29334 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:02.289001 29090 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.249764 29007 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:02.296517 25706 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:02.202406 29362 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.353075 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.329214 25709 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.252661 29329 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:02.362871 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:02.363793 26335 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:02.367542 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.368790 29334 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:02.369275 29334 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.369680 29334 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:19:02.306883 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.256916 29251 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.383589 29251 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.351310 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:02.349073 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.386950 25705 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:02.387140 29393 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:02.387567 29251 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.388080 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.389194 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.389272 29393 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:02.389866 25705 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:02.390767 29394 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:02.390872 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.392062 25705 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:02.391883 29391 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:02.393044 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.390400 29393 consensus_queue.cc:1243] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:02.293675 29035 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:19:02.288467 29191 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:02.413370 29191 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:02.413928 29191 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:02.410641 29329 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.419556 29022 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:02.420382 29022 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.420930 29022 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.419535 29329 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.424680 29022 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:02.405735 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.411938 28983 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.411540 29393 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:02.425607 28983 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.424922 29191 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.428876 28983 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.430785 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:02.430642 29191 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:02.432637 25783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:02.433264 25783 raft_consensus.cc:606] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.433620 25783 raft_consensus.cc:614] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:02.438623 25710 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:02.441957 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:02.442271 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:02.443084 29329 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:02.443480 26339 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:02.444562 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:02.300607 29383 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.145s
I20260709 12:19:02.443540 29329 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.445921 29329 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:02.447232 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.447873 25706 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:19:02.448737 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.449429 25706 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:02.356066 29399 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:19:02.300446 29213 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:02.456184 29213 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.456558 29213 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:02.459322 29396 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:02.461691 29362 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.463349 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:02.463958 26339 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:02.405069 29035 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:02.290421 29029 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:19:02.466596 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.293452 29337 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.480345 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.462832 29329 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.481730 29321 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:02.482753 29350 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:02.483327 29350 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:02.483762 29352 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:02.484242 29350 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.487253 29294 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:02.403203 29339 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.500515 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.501828 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:02.504366 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:02.477427 26337 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:02.470458 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.470887 26335 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:02.479301 29035 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:19:02.473618 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:02.498723 29321 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:19:02.520195 26339 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:02.553294 25706 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:19:02.504968 29303 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:02.570200 25783 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:02.555805 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51523.2us
I20260709 12:19:02.565562 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.587778 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:02.568902 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:02.562045 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:02.592872 26344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 884) took 1257 ms (client timeout 1673 ms). Trace:
I20260709 12:19:02.598151 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.600029 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:02.593416 26344 rpcz_store.cc:269] 0709 12:19:01.335810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.335994 (+   184us) service_pool.cc:224] Handling call
0709 12:19:02.584256 (+1248262us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b540058ff54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:02.592785 (+  8529us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848710016}
I20260709 12:19:02.505771 29339 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.538441 29329 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.575775 29331 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:02.576922 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:02.614822 25782 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:02.578050 29004 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:02.503490 29298 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.618415 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:02.579218 29337 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.579335 29090 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:02.579838 25786 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:19:02.582530 29245 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:02.620705 29245 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:02.506735 29334 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.592468 29303 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:02.564270 29294 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:02.612658 28983 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:02.619490 25750 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.583324 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 775) took 1092 ms. Trace:
I20260709 12:19:02.632158 25629 rpcz_store.cc:276] 0709 12:19:01.490419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.809007 (+318588us) service_pool.cc:224] Handling call
0709 12:19:02.583074 (+774067us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b54001d6fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:02.583234 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88777472}
I20260709 12:19:02.634791 29337 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:02.637038 29334 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:02.638680 25779 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:02.639029 29294 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:02.640107 29294 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:02.641090 29325 consensus_meta.cc:287] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.000s
W20260709 12:19:02.642144 29397 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:02.642655 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1805) took 1262 ms. Trace:
I20260709 12:19:02.643054 25633 rpcz_store.cc:276] 0709 12:19:01.380021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.380195 (+   174us) service_pool.cc:224] Handling call
0709 12:19:02.070666 (+690471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.071415 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.642412 (+570997us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b540023fd50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:02.642591 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482416384}
I20260709 12:19:02.644528 25631 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:19:02.622644 25706 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:02.610648 29007 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:19:02.568816 29330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.297s	user 0.044s	sys 0.056s
I20260709 12:19:02.626793 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:02.611898 29365 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:19:02.661113 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 269190us
W20260709 12:19:02.663622 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 336298us
W20260709 12:19:02.653373 29330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.400s	user 0.045s	sys 0.056s
I20260709 12:19:02.653892 26339 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:02.656647 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.677269 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1766) took 1831 ms (client timeout 1660 ms). Trace:
I20260709 12:19:02.652200 25781 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:02.677748 25787 rpcz_store.cc:269] 0709 12:19:00.845745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.850583 (+  4838us) service_pool.cc:224] Handling call
0709 12:19:02.677046 (+1826463us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000de3d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:02.677185 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460243200}
I20260709 12:19:02.572796 25709 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
W20260709 12:19:02.679019 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.679463 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.682627 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:02.658097 29365 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.687367 25709 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:19:02.658226 28983 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:02.698614 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.650947 29213 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.704464 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.706874 26339 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:02.708124 25750 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:02.709622 29281 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:02.700771 29258 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.710800 29281 consensus_queue.cc:1243] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:19:02.711491 29281 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:19:02.712649 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.682209 29391 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.702824 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.717300 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:02.702651 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:02.730355 29221 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:02.730850 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:02.730998 29221 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:02.721231 25781 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:02.732868 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.678335 29330 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:02.733518 25781 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:02.734637 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:02.734826 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:02.735368 25781 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:02.684129 29007 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.631204 29245 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.690292 29365 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:02.732589 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:02.735255 25637 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:19:02.742607 25637 pending_rounds.cc:85] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 7
I20260709 12:19:02.699842 25709 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:02.711989 29281 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:02.740043 25603 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.736268 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.735958 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:02.738761 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:02.741991 26342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:02.743052 25637 pending_rounds.cc:107] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:19:02.746553 25637 raft_consensus.cc:2889] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:02.699098 29004 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:02.794395 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:02.806447 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 66487.1us
I20260709 12:19:02.808743 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:02.770541 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:02.809800 25787 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:02.729673 29347 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:02.739147 29191 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:02.737104 29258 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.736516 29406 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:19:02.834105 25677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.016s	sys 0.021s
W20260709 12:19:02.834728 25677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.017s	sys 0.021s
I20260709 12:19:02.732520 29221 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.836745 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:02.747821 29333 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:19:02.841188 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:02.711612 29303 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.742079 29383 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:02.744786 29283 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.740165 29391 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:02.853720 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:02.856518 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:02.858685 25638 raft_consensus.cc:2410] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:19:02.830771 29191 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:02.734166 29330 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.743026 29291 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:19:02.807857 29362 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:19:02.863950 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:02.751052 29036 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:02.858965 25709 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:02.751477 29007 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.817102 29004 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:02.827443 29029 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:19:02.850425 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:02.894773 26342 raft_consensus.cc:606] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:02.904835 26342 raft_consensus.cc:614] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:02.855027 29406 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:02.897378 25782 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:02.906004 26339 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:02.894264 25630 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:02.940546 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.885200 29191 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:02.876423 29383 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:02.831050 29347 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:02.976055 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 273462us
I20260709 12:19:02.971699 25707 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:19:02.990667 25782 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:19:02.991457 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:02.992579 25779 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:02.992892 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:02.896127 29036 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:02.886684 29107 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.019s	sys 0.014s
I20260709 12:19:02.899971 29329 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:02.886694 29183 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:02.906297 29333 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.992110 25707 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:02.839962 28983 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:03.015089 29006 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.808s	user 0.012s	sys 0.000s
I20260709 12:19:03.015645 29006 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:03.016829 29183 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.018667 29333 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.020910 29333 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.021416 29416 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.012847 29404 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:19:03.021894 25678 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.012s	sys 0.016s
I20260709 12:19:03.013027 29329 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.013262 29417 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:03.013880 29413 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:02.987043 29259 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:03.023191 25678 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.012s	sys 0.016s
I20260709 12:19:02.870081 29283 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.014036 29339 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:02.839450 29281 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:03.057343 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 349831us
I20260709 12:19:03.071326 25707 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:19:03.006222 29107 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.019s	sys 0.014s
I20260709 12:19:03.001253 29036 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:19:03.033869 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 221288us
I20260709 12:19:03.054381 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:03.072871 25708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.019s	sys 0.043s
W20260709 12:19:03.101780 25708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.020s	sys 0.043s
I20260709 12:19:03.026963 29417 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.029028 29213 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:03.112811 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.113792 25707 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:03.107358 25637 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
W20260709 12:19:03.108037 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 369357us
W20260709 12:19:03.118034 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.120823 25709 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:19:03.120806 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:03.102399 29405 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.390s	user 0.001s	sys 0.005s
I20260709 12:19:03.004169 29411 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
W20260709 12:19:03.123795 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 131359us
I20260709 12:19:03.130002 26339 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:03.130151 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2262) took 1145 ms. Trace:
I20260709 12:19:03.133347 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:03.134253 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:03.133311 25785 rpcz_store.cc:276] 0709 12:19:01.984585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.655190 (+670605us) service_pool.cc:224] Handling call
0709 12:19:02.672505 (+ 17315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.672725 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:19:02.737370 (+ 64645us) log.cc:844] Serialized 14 byte log entry
0709 12:19:03.129523 (+392153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:03.129715 (+   192us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:03.129797 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.129889 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.130075 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65217,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":3841,"raft.queue_time_us":21634,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":371611,"threads_started":1}
I20260709 12:19:03.136356 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:03.024469 29416 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.053438 29259 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.107510 29417 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.176795 29183 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.178135 29183 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.183104 29258 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.190842 25630 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:03.132309 29294 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:03.197978 25786 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:19:03.199968 29442 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:03.200590 29442 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:03.201351 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.202068 25706 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:03.202461 29007 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.202077 29442 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.220521 25707 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
W20260709 12:19:03.210898 29035 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 12:19:03.225047 29294 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:03.242677 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.242383 25706 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
W20260709 12:19:03.259198 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.250586 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:03.193166 29414 raft_consensus.cc:2764] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:19:03.225627 29333 raft_consensus.cc:2764] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:19:03.274665 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:03.276293 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:03.280879 26344 raft_consensus.cc:2368] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
W20260709 12:19:03.283082 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 159688us
I20260709 12:19:03.284269 26339 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:03.112543 29383 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.290992 26344 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:19:03.206326 29183 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.230235 29221 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:19:03.294028 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.299470 25706 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:03.269170 29436 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:03.300686 25706 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:03.281773 29004 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.273052 29334 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:03.280334 29413 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.295600 25713 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:19:03.258349 29035 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.301667 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.314893 29416 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.301347 29436 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:03.301275 29411 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:03.315893 29436 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.316049 29411 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.301069 29221 consensus_queue.cc:1243] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:03.286883 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2350) took 1046 ms. Trace:
I20260709 12:19:03.308290 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:03.334167 29430 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:03.334831 29430 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.335755 29334 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:03.337594 29035 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.336818 29417 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.339833 25636 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:03.347960 29036 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.337222 29430 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.355218 29445 raft_consensus.cc:476] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Not starting pre-election -- already a leader
I20260709 12:19:03.356699 29417 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:03.341413 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 207185us
I20260709 12:19:03.360538 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:03.330945 25708 rpcz_store.cc:276] 0709 12:19:02.240639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.240747 (+   108us) service_pool.cc:224] Handling call
0709 12:19:02.241064 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.241293 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:02.546040 (+304747us) log.cc:844] Serialized 14 byte log entry
0709 12:19:03.286552 (+740512us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:03.286605 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:03.286647 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.286726 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.286830 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14591,"prepare.queue_time_us":389377,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":7674,"raft.queue_time_us":94696,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":13654,"spinlock_wait_cycles":176401792,"thread_start_us":1044304,"threads_started":3,"wal-append.queue_time_us":652489}
I20260709 12:19:03.295048 29298 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:19:03.361743 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:03.327543 29365 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.000s
I20260709 12:19:03.350274 29036 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.367831 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:03.326882 29329 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:03.370246 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 85635.4us
I20260709 12:19:03.363175 25636 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:03.323397 29259 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:03.365840 29348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.167s	user 0.009s	sys 0.006s
I20260709 12:19:03.328504 29294 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.376132 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.379212 25630 pending_rounds.cc:85] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 6
I20260709 12:19:03.382690 25630 pending_rounds.cc:107] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:19:03.383004 25630 raft_consensus.cc:2889] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:03.363758 29035 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.387328 29035 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.387852 29183 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.378378 29406 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 12:19:03.389150 29440 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:19:03.365569 29417 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.389770 29035 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.390058 29417 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.391455 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2260) took 1413 ms. Trace:
W20260709 12:19:03.391748 29419 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:19:03.392323 29417 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.391882 25779 rpcz_store.cc:276] 0709 12:19:01.978063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.639400 (+661337us) service_pool.cc:224] Handling call
0709 12:19:02.950434 (+311034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.993191 (+ 42757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.381560 (+388369us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540011fb50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:03.391377 (+  9817us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714609536}
I20260709 12:19:03.393616 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2348) took 1153 ms. Trace:
I20260709 12:19:03.393888 26341 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:03.394019 25711 rpcz_store.cc:276] 0709 12:19:02.240078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.248599 (+  8521us) service_pool.cc:224] Handling call
0709 12:19:02.260394 (+ 11795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.295927 (+ 35533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.295974 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.301959 (+  5985us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.876319 (+574360us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:02.876363 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.876435 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.391103 (+514668us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:03.393317 (+  2214us) raft_consensus.cc:1685] finished
0709 12:19:03.393343 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.393547 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":581,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":56704,"spinlock_wait_cycles":281864960,"thread_start_us":429286,"threads_started":1,"wal-append.queue_time_us":698519}
W20260709 12:19:03.379092 29348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.010s	sys 0.006s
I20260709 12:19:03.400098 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:03.327143 29432 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:19:03.401983 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:03.411141 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.423460 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:03.424446 29417 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:03.378006 29347 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.401724 29329 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:03.425889 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:03.324975 29413 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.437505 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:03.441524 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 56108.3us
I20260709 12:19:03.427444 29417 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:03.452431 29449 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:03.452908 29449 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:03.453277 29449 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:03.455471 25635 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:03.456059 25635 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:03.457129 25635 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:03.467727 29451 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:19:03.470291 25635 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.373668 29365 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.427222 29450 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:03.401302 29348 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:03.488771 29348 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.490334 29191 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.493153 29191 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.430265 29183 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:19:03.374871 29334 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.496778 25634 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:19:03.497792 25631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2143) took 2757 ms (client timeout 2667 ms). Trace:
W20260709 12:19:03.498291 25631 rpcz_store.cc:269] 0709 12:19:00.739942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.973303 (+233361us) service_pool.cc:224] Handling call
0709 12:19:03.496167 (+2522864us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400300cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:03.497667 (+  1500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3550350592}
I20260709 12:19:03.499650 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.501613 25631 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:03.420264 29035 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:03.503703 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 77917.2us
I20260709 12:19:03.434788 29347 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:03.440603 29283 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.514706 25635 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 12:19:03.445847 29329 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:03.520443 29454 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:03.524667 25601 leader_election.cc:400] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6.
I20260709 12:19:03.526618 25601 leader_election.cc:403] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:19:03.528162 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:03.535033 26338 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:03.539187 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:03.530809 29283 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6.
I20260709 12:19:03.545564 25631 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:03.545593 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:03.536506 29417 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:03.547465 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:03.548887 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:03.548976 29417 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:03.550987 29006 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:19:03.552698 29407 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.469s	user 0.000s	sys 0.003s
W20260709 12:19:03.554630 29423 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.370s	user 0.001s	sys 0.001s
I20260709 12:19:03.554979 29413 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.538004 29437 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:03.558693 29036 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:03.546326 29450 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:03.541363 29365 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.550041 29417 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.566646 29036 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.574155 29258 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:03.586596 29420 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.574s	user 0.004s	sys 0.000s
I20260709 12:19:03.588402 25601 leader_election.cc:400] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
I20260709 12:19:03.588807 25601 leader_election.cc:403] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:03.589154 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:03.589618 25785 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:03.590011 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.590440 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:03.590710 29441 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:19:03.594920 29441 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:03.592276 25785 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:03.594805 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:03.595710 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:03.597254 25784 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:03.598640 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:03.599498 25779 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:03.591253 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:03.601095 25778 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:19:03.576220 29450 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:03.604053 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.580794 29036 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:03.588587 29410 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.594s	user 0.000s	sys 0.007s
I20260709 12:19:03.571835 29006 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:19:03.595525 29441 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:03.563428 29437 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:03.618122 29421 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.367s	user 0.000s	sys 0.002s
W20260709 12:19:03.619467 29422 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.420s	user 0.006s	sys 0.000s
W20260709 12:19:03.620565 29261 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:19:03.627810 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.626226 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:03.611120 29283 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:19:03.640241 25638 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:03.641110 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:03.615531 29432 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.626520 29258 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:03.649585 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.647993 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:03.643769 29417 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:03.660446 25638 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:03.627067 29441 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.668202 29441 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:03.668241 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.669078 29417 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.669543 29417 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.652019 29393 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:03.648639 29258 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:03.656446 29261 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:03.676548 29449 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.652179 29320 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:19:03.629786 29450 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.695165 26344 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:03.686906 29393 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.696224 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:03.696852 26335 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:03.699070 26343 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:03.700577 29322 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.679548 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.703855 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.704447 25635 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:03.705228 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.683393 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2255) took 1742 ms. Trace:
I20260709 12:19:03.706622 25783 rpcz_store.cc:276] 0709 12:19:01.940968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.434949 (+493981us) service_pool.cc:224] Handling call
0709 12:19:02.435240 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.573452 (+138212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.683137 (+1109685us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400258ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:03.683330 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294972288}
I20260709 12:19:03.694245 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:03.686033 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2258) took 1727 ms. Trace:
I20260709 12:19:03.711342 25786 rpcz_store.cc:276] 0709 12:19:01.958759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.623187 (+664428us) service_pool.cc:224] Handling call
0709 12:19:02.623560 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.205385 (+581825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.685786 (+480401us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b540030af54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:03.685974 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320237312}
I20260709 12:19:03.629288 29436 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.687532 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.690371 29258 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.691305 29449 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.696345 29393 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:03.679026 29437 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:03.728639 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.729251 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.648763 29432 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:03.729607 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.729979 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.706328 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.722245 29035 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:03.729303 29006 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.733929 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.734102 29006 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.734696 25635 raft_consensus.cc:2368] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:19:03.689045 29261 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.743858 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.724877 29258 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:03.674968 29325 consensus_queue.cc:1175] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "10d0c7349085463595aedaf7c071e742" responder_term: 7 status { last_received { term: 6 index: 5 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:03.732542 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:03.730288 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.751008 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.751246 25603 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:19:03.722831 29435 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.436s	user 0.005s	sys 0.000s
I20260709 12:19:03.743093 25637 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:03.744035 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.720413 29315 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:03.746398 29006 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.746837 29291 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:03.761159 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1779) took 1074 ms. Trace:
I20260709 12:19:03.759624 29320 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:03.747200 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.762478 29320 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:03.761557 25787 rpcz_store.cc:276] 0709 12:19:02.686520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.890039 (+203519us) service_pool.cc:224] Handling call
0709 12:19:02.904980 (+ 14941us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.905336 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.905364 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.905821 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.125524 (+219703us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:03.125593 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.125683 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.722146 (+596463us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:03.760881 (+ 38735us) raft_consensus.cc:1685] finished
0709 12:19:03.760903 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.761102 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2118,"raft.queue_time_us":43662,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5602,"spinlock_wait_cycles":14778240,"thread_start_us":124470,"threads_started":1,"wal-append.queue_time_us":234947}
I20260709 12:19:03.763327 29320 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:03.763794 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.764451 25713 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:03.747819 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.765308 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.765995 25635 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:03.755007 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.767129 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.769083 25705 raft_consensus.cc:2368] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6.
I20260709 12:19:03.771016 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:03.756807 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:03.759553 28983 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:03.760075 29432 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.754647 29315 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:03.748118 29261 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.775401 29035 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:03.779461 25603 leader_election.cc:400] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6.
I20260709 12:19:03.779305 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.799835 25603 leader_election.cc:403] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:19:03.781765 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.773231 29437 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.784796 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.783203 29006 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:03.780597 29430 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.789052 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.790010 29436 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.792075 29417 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.800128 25630 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:03.802716 25708 raft_consensus.cc:2410] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:19:03.785142 29315 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:03.806438 25635 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:03.802965 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1804) took 2416 ms. Trace:
I20260709 12:19:03.808372 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.810688 25634 rpcz_store.cc:276] 0709 12:19:01.379600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.413093 (+ 33493us) service_pool.cc:224] Handling call
0709 12:19:03.496321 (+2083228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.497264 (+   943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.795902 (+298638us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b5400300cd0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:03.796081 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4790082944}
I20260709 12:19:03.816939 29417 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.808828 29430 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.820914 29430 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.823195 25630 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9.
I20260709 12:19:03.823680 29357 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:03.824198 29357 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:03.825055 29357 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.827823 29357 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:03.828099 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.804339 29437 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.825064 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 640) took 1844 ms. Trace:
I20260709 12:19:03.835222 26338 rpcz_store.cc:276] 0709 12:19:01.980833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.981020 (+   187us) service_pool.cc:224] Handling call
0709 12:19:01.994062 (+ 13042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.535511 (+1541449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.824792 (+289281us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54000a2b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:03.824974 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3544082048}
I20260709 12:19:03.804814 29006 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.807269 29441 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6.
I20260709 12:19:03.850457 25712 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:03.850946 25787 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:03.855007 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.856879 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:03.860953 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:03.861924 25780 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.863729 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.864602 25635 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:03.823680 29328 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:03.885764 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:03.848266 29006 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:03.889895 29477 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:19:03.890240 29337 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.854169 29090 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.891309 29437 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.915977 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.917102 25780 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:03.918650 29221 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:03.917151 25637 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:19:03.918916 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:03.919642 29221 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.920318 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.921746 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.921913 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:03.922484 25637 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:03.923341 29221 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.927588 29452 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:03.928145 29452 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.928522 29452 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:03.931149 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 779) took 1201 ms. Trace:
I20260709 12:19:03.931630 25633 rpcz_store.cc:276] 0709 12:19:02.729295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.731788 (+  2493us) service_pool.cc:224] Handling call
0709 12:19:02.732080 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.988375 (+256295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.988422 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.988475 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:02.988513 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.930873 (+942360us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400363154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:03.930929 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.931082 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588785536}
I20260709 12:19:03.933558 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:03.883488 29328 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:03.939744 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:03.946064 25637 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:03.899029 29441 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.896883 29352 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:03.854331 28983 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:03.958878 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:03.980795 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:03.950896 29322 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:03.906131 29090 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.938972 29036 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.992077 29036 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.994856 29261 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:03.997267 29411 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.997562 29261 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:03.962260 29340 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.002403 29340 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.005084 29340 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:03.989609 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:04.006642 25708 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:04.001673 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 798) took 1300 ms. Trace:
I20260709 12:19:04.007601 26341 rpcz_store.cc:276] 0709 12:19:02.701326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.701456 (+   130us) service_pool.cc:224] Handling call
0709 12:19:03.393536 (+692080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.394466 (+   930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.001399 (+606933us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b540061c450. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:04.001598 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590984448}
I20260709 12:19:03.906256 29337 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.009431 29391 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.011888 29337 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.013137 25634 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:04.016762 29404 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:04.017218 29404 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:04.018064 29404 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:03.962595 29299 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:03.953070 29436 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:19:03.864928 29291 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.150s
I20260709 12:19:03.982175 29283 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
I20260709 12:19:03.987483 29413 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:03.987249 29459 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:03.975292 29441 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.038189 25631 raft_consensus.cc:2410] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:19:04.028359 29331 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.008s	sys 0.020s
I20260709 12:19:03.983712 28983 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:04.034437 29442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.021s	sys 0.024s
W20260709 12:19:04.042105 29331 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.008s	sys 0.020s
I20260709 12:19:04.075269 25638 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:04.043524 29413 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:04.160112 29413 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.161110 29006 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.028035 29299 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:04.053800 29442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.022s	sys 0.024s
I20260709 12:19:04.162897 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:04.163502 29299 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.026387 29328 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.163870 26336 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:04.164084 29484 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:04.164628 29484 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.164983 29484 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:04.098939 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57550 (request call id 1) took 1408 ms. Trace:
I20260709 12:19:04.166662 26342 rpcz_store.cc:276] 0709 12:19:02.699881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.712440 (+ 12559us) service_pool.cc:224] Handling call
0709 12:19:04.095346 (+1382906us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54006caf54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5989853 00007f29ba95760a 00007f29b3edd7d4 00007f29b3edd762 00007f29b3edd691 00007f29b3edd60c 00007f29b3edc8c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:04.095528 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341359488}
W20260709 12:19:04.121366 29443 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.919s	user 0.002s	sys 0.002s
I20260709 12:19:04.168115 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:04.131191 29474 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
I20260709 12:19:04.169890 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1785) took 1426 ms. Trace:
I20260709 12:19:04.132314 25603 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:04.134199 29475 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:19:04.138561 28983 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:04.179081 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 293402us
I20260709 12:19:04.180958 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:04.139986 29281 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:04.094547 29393 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.148723 29441 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.151185 29321 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:04.151691 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:04.183883 25713 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:04.151921 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:04.152612 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:04.185806 25707 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:04.186599 26342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:04.161223 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:04.187752 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.170989 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.170367 25711 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:04.189213 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:04.192589 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:04.170295 25782 rpcz_store.cc:276] 0709 12:19:02.743032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.991464 (+248432us) service_pool.cc:224] Handling call
0709 12:19:03.006979 (+ 15515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.023861 (+ 16882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.023903 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.024366 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.167414 (+143048us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:03.167462 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.168112 (+   650us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.120699 (+952587us) spinlock_profiling.cc:243] Waited 88.7 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:04.169606 (+ 48907us) raft_consensus.cc:1685] finished
0709 12:19:04.169628 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.169829 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26132,"raft.queue_time_us":45793,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":46794,"spinlock_wait_cycles":204104064,"thread_start_us":63699,"threads_started":1,"wal-append.queue_time_us":145863}
I20260709 12:19:04.164480 29328 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:04.174773 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 784) took 1444 ms. Trace:
I20260709 12:19:04.119948 25634 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:19:04.195990 25636 rpcz_store.cc:276] 0709 12:19:02.730810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.735730 (+  4920us) service_pool.cc:224] Handling call
0709 12:19:04.174462 (+1438732us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b54001e91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:04.174589 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1365403520}
W20260709 12:19:04.176919 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44724.4us
I20260709 12:19:04.177733 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:04.097791 29479 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:19:04.193001 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:04.200999 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.157572 29452 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.202647 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:04.205910 25631 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
W20260709 12:19:04.203727 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.330214 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.331285 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.331540 26342 raft_consensus.cc:606] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.204483 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:04.204807 26309 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.234970 29348 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:19:04.338312 25708 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:04.206132 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.282833 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:04.336376 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 131634us
I20260709 12:19:04.344261 25634 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:04.295197 25713 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:04.202298 29337 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:19:04.341961 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.309669 25707 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
W20260709 12:19:04.334965 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 154119us
I20260709 12:19:04.344492 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.195875 29415 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.234046 29453 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:19:04.355149 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.305059 29430 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.198870 28983 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.359040 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.350211 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.205652 29454 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:04.389948 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:04.405280 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46360.4us
I20260709 12:19:04.341068 29334 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:04.398548 29283 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:04.403882 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 53747.7us
I20260709 12:19:04.414022 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:04.412027 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.351456 29261 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.416098 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.418170 25751 leader_election.cc:400] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7.
I20260709 12:19:04.420452 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:04.427991 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.403483 29330 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:04.411945 29337 raft_consensus.cc:2764] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:19:04.410840 29415 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:04.403458 29453 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:04.433130 29441 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:04.420327 29476 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:19:04.413707 29348 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.403998 29430 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.422909 29495 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:04.379081 29414 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:19:04.419426 29258 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:04.435683 29459 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:04.432374 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.409528 28983 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.433290 29291 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.434955 29330 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.416389 29322 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:04.457600 29320 consensus_meta.cc:287] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.689s	user 0.013s	sys 0.002s
I20260709 12:19:04.423369 29283 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.423094 29261 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:04.478180 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45884.5us
I20260709 12:19:04.486845 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 536) took 2126 ms. Trace:
I20260709 12:19:04.496868 25709 rpcz_store.cc:276] 0709 12:19:02.360039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.896955 (+536916us) service_pool.cc:224] Handling call
0709 12:19:02.897347 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.121401 (+224054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.486534 (+1365133us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54002238d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:04.486773 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513354112}
I20260709 12:19:04.467250 29006 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.458331 29323 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:19:04.469362 29459 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:04.511545 29459 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:04.490192 29337 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:04.515676 29357 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.679s	user 0.008s	sys 0.004s
I20260709 12:19:04.519251 29283 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.520583 29383 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.521395 29383 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.465238 29348 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.523358 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.524277 26336 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
W20260709 12:19:04.525616 29494 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:04.529140 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:04.530633 29337 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:04.531036 29337 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:04.531152 29489 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:04.531437 29337 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:04.533928 29452 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.534880 25636 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:19:04.535650 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 181642us
I20260709 12:19:04.507227 29415 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 12:19:04.539686 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 113802us
I20260709 12:19:04.511039 28983 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.487421 29393 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.541922 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.540455 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.490741 29330 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:04.544255 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.551831 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.553359 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.550695 29499 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:04.555315 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.553973 26341 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:19:04.556272 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:04.559659 29425 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.383s	user 0.007s	sys 0.000s
I20260709 12:19:04.559568 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.561007 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 531) took 2295 ms. Trace:
I20260709 12:19:04.561441 25704 rpcz_store.cc:276] 0709 12:19:02.266077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.651335 (+385258us) service_pool.cc:224] Handling call
0709 12:19:02.667180 (+ 15845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.667521 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.667560 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.683725 (+ 16165us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.024692 (+340967us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:03.024730 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.024805 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.559072 (+1534267us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:04.560735 (+  1663us) raft_consensus.cc:1685] finished
0709 12:19:04.560754 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.560946 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274368384,"thread_start_us":280250,"threads_started":1,"wal-append.queue_time_us":398884}
I20260709 12:19:04.563714 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:04.564399 25638 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:04.565106 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.565863 25638 raft_consensus.cc:2410] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:19:04.566736 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.567277 25638 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:04.568217 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.568787 25638 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:04.550503 29415 consensus_queue.cc:1243] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:04.502070 29322 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.496177 29320 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.571043 26338 raft_consensus.cc:2393] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:04.551937 29298 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:19:04.550580 29334 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:04.573371 26341 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:19:04.577586 26341 pending_rounds.cc:85] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 6
I20260709 12:19:04.577848 26341 pending_rounds.cc:107] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:19:04.578068 26341 raft_consensus.cc:2889] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:04.587546 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.588699 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:04.590528 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.591971 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.592753 25638 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:04.594166 25638 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:04.571977 29495 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.611068 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.547434 29444 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:19:04.574014 29499 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:04.613477 26344 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:04.615316 26344 raft_consensus.cc:1217] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 6.7->[7.8-7.8]   Dedup: 7.8->[]
I20260709 12:19:04.615877 26344 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:19:04.617319 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:04.620757 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 134546us
I20260709 12:19:04.538065 29453 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.627678 25705 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:04.629851 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.623582 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:04.632694 26335 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:04.633961 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:04.634100 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 135383us
W20260709 12:19:04.634819 25751 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.439s (SENT)
I20260709 12:19:04.636595 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.637552 25778 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:04.637300 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.638454 25711 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:19:04.639772 25713 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:19:04.641665 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:04.642879 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:04.601141 29334 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:04.606266 29298 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:04.644809 25711 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:19:04.645589 25710 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:04.649533 25782 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:19:04.654580 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:04.658142 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:04.663717 25704 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:04.667092 29251 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:04.667560 29251 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:04.645382 29418 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.469s	user 0.003s	sys 0.003s
I20260709 12:19:04.677074 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:04.667887 29251 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:04.612743 29495 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.681818 29452 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:19:04.693404 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.683781 25638 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:04.647593 29331 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:04.664026 29291 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.679391 29383 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.705047 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:04.705965 25778 raft_consensus.cc:2376] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6. Re-sending same reply.
I20260709 12:19:04.710286 25677 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:04.710664 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.712097 25677 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.713198 25752 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.717623 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:04.719278 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.701850 29331 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.722661 29436 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:19:04.723018 29315 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.894s	user 0.015s	sys 0.000s
I20260709 12:19:04.723224 29436 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:04.723420 29315 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.723687 29436 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:04.723716 29331 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.725504 29315 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.726547 29508 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:04.726953 29508 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:04.727312 29508 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:04.727535 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:04.728394 25637 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:04.729617 25637 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:19:04.731426 25635 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:04.731781 25635 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:04.732156 29251 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.732664 25635 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.734015 25635 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:04.734704 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.735055 29320 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.735179 29499 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.735589 29315 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:04.736440 29315 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:04.737017 29315 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:04.740207 29322 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.704875 29261 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:04.742901 29328 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 12:19:04.703030 29291 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.744561 25783 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:04.745607 29291 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:04.746793 29298 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.748605 25782 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:04.750864 25637 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:19:04.752391 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.755235 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.759189 25705 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:04.747431 29251 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:04.792240 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:04.775632 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.793327 25781 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:04.776309 25778 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:04.735072 29502 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:19:04.720441 29452 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:04.780325 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:04.781088 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:04.778195 29281 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.004s
I20260709 12:19:04.788389 29508 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.789522 25636 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:19:04.790163 29291 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:04.763664 25637 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:19:04.789968 29330 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.802668 29251 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:04.798389 29455 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:04.814119 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2179) took 1229 ms. Trace:
I20260709 12:19:04.823697 29495 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:04.750752 29348 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:04.838411 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 214906us
I20260709 12:19:04.840656 25631 rpcz_store.cc:276] 0709 12:19:03.584359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.124351 (+539992us) service_pool.cc:224] Handling call
0709 12:19:04.124630 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.329201 (+204571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.813735 (+484534us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b54001c61d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:04.814038 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171312896}
I20260709 12:19:04.842299 25637 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:19:04.843483 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:04.846971 25635 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:19:04.847319 25635 pending_rounds.cc:85] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 6
I20260709 12:19:04.847527 25635 pending_rounds.cc:107] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:19:04.847800 25635 raft_consensus.cc:2889] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:04.844523 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:04.790566 29334 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.850407 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.840116 29251 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:04.850973 25712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2399) took 3861 ms (client timeout 2076 ms). Trace:
I20260709 12:19:04.858300 25782 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:04.849999 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.860049 25786 raft_consensus.cc:2410] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:19:04.857777 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:04.854146 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.865552 25787 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:04.827551 29391 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:04.858580 25712 rpcz_store.cc:269] 0709 12:19:00.983627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.983776 (+   149us) service_pool.cc:224] Handling call
0709 12:19:04.844693 (+3860917us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b5400104354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:04.844813 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6418201472}
I20260709 12:19:04.867183 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:04.868124 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2185) took 1025 ms. Trace:
I20260709 12:19:04.869463 26341 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:04.877692 25635 rpcz_store.cc:276] 0709 12:19:03.842619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.730987 (+888368us) service_pool.cc:224] Handling call
0709 12:19:04.731276 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.867405 (+136129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.867766 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:04.838878 29430 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.881072 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:04.881731 25635 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:04.867342 29251 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:04.867223 29429 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:19:04.876590 29321 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:19:04.848927 29455 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.887429 29321 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:04.887902 29321 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:04.888880 29455 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.889015 29321 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.892048 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:04.892771 25786 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:04.894344 29508 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:04.894562 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.867107 29452 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 12:19:04.896719 29498 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.343s	user 0.002s	sys 0.000s
I20260709 12:19:04.897531 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.898202 25782 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:04.898821 29444 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:04.899749 29508 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:04.900259 29508 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:04.901556 29508 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.903028 25781 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:19:04.896656 29452 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:04.908496 29452 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:04.909646 29452 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:04.911139 29452 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:04.920024 25784 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:04.899844 29391 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:04.931308 25781 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:19:04.923041 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.923309 25786 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:04.889655 29455 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:19:04.935091 29492 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.475s	user 0.002s	sys 0.000s
W20260709 12:19:04.939038 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.942863 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 337935us
I20260709 12:19:04.949256 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:04.950563 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:04.922112 29321 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:04.992383 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69437.5us
I20260709 12:19:05.022068 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:05.024674 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:05.026250 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:04.987444 29454 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.023s
I20260709 12:19:04.949335 29455 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:04.954941 29348 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:19:04.921209 29487 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 12:19:05.045038 25781 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:05.032346 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:04.994642 29444 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:04.956638 29245 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:04.995359 29437 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:05.060132 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:05.096191 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:05.100862 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 47982.1us
W20260709 12:19:05.095285 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.202451 29325 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:05.242174 25705 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:19:05.244774 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:05.245539 25705 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:05.204726 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:05.246169 25781 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:05.202720 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 170470us
I20260709 12:19:05.034988 29415 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:04.995761 29393 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:05.088955 29294 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.139667 29454 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.004s	sys 0.028s
I20260709 12:19:04.997897 29413 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.248243 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:05.315526 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57550 (request call id 2) took 1129 ms. Trace:
I20260709 12:19:05.316098 26342 rpcz_store.cc:276] 0709 12:19:04.186230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.186391 (+   161us) service_pool.cc:224] Handling call
0709 12:19:05.315318 (+1128927us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54004970d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5989853 00007f29ba95760a 00007f29b3edd7d4 00007f29b3edd762 00007f29b3edd691 00007f29b3edd60c 00007f29b3edc8c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:05.315447 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331267968}
I20260709 12:19:05.178675 25525 maintenance_manager.cc:419] P 3c2fa83ead874145ba497a2c6d2a12d0: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033378
W20260709 12:19:05.251590 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 155524us
I20260709 12:19:05.253520 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2172) took 1671 ms. Trace:
W20260709 12:19:05.256083 25632 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.030s	sys 0.058s
W20260709 12:19:05.334698 25632 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.030s	sys 0.058s
I20260709 12:19:05.334218 25633 rpcz_store.cc:276] 0709 12:19:03.582382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.953121 (+370739us) service_pool.cc:224] Handling call
0709 12:19:03.977909 (+ 24788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.777709 (+799800us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:04.777744 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.777791 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:04.777825 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.253311 (+475486us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540020b7d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:05.253357 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.253462 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33841,"raft.queue_time_us":34401,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":11119,"spinlock_wait_cycles":1760621312}
I20260709 12:19:05.250288 29415 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:05.040792 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 405793us
W20260709 12:19:05.283099 26337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.025s
I20260709 12:19:05.239336 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:05.346765 26337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.008s	sys 0.025s
I20260709 12:19:05.347215 25712 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:05.247330 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:04.993451 29436 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:05.300870 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:05.291661 29357 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.292122 26343 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.008s	sys 0.151s
W20260709 12:19:05.360210 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.009s	sys 0.017s
W20260709 12:19:05.360339 26343 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.009s	sys 0.151s
W20260709 12:19:05.360675 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.009s	sys 0.017s
I20260709 12:19:05.321044 29454 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:05.302191 29036 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.311273 29455 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.313664 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:05.325652 25709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.009s	sys 0.031s
I20260709 12:19:05.331537 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:19:05.287187 29375 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.288479 26338 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:05.301760 25704 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:05.291217 29507 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:05.355183 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:05.356464 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.358870 25712 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:05.362370 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.346040 29415 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:05.350901 29298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.068s	sys 0.101s
I20260709 12:19:05.369413 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:05.445331 25633 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:05.047283 29484 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.876s	user 0.007s	sys 0.007s
W20260709 12:19:05.361027 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 517057us
I20260709 12:19:05.461980 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:05.445677 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:05.447818 25630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2158) took 1868 ms (client timeout 2439 ms). Trace:
I20260709 12:19:05.448662 25631 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:05.211695 29444 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.462836 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:05.467900 25630 rpcz_store.cc:269] 0709 12:19:03.579220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.808089 (+228869us) service_pool.cc:224] Handling call
0709 12:19:05.447488 (+1639399us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b54001c3ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:05.447732 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25381760}
I20260709 12:19:05.333753 29413 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:05.485345 25709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.009s	sys 0.032s
I20260709 12:19:05.482966 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.489254 25784 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:05.490623 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:05.491415 25784 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:05.483770 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.484082 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:05.318251 29294 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.345299 29325 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
W20260709 12:19:05.459785 29298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.070s	sys 0.103s
I20260709 12:19:05.504724 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:05.434211 29507 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.515318 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:05.516773 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:05.530975 25781 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 12:19:05.536585 25633 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:05.535797 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:05.538815 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:05.539511 25705 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:05.540768 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:05.534855 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:05.542732 25707 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:05.455082 29322 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:05.465601 29281 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:05.534803 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:05.471958 29330 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:05.546943 25707 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:05.552069 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 801) took 1355 ms. Trace:
I20260709 12:19:05.477697 29444 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:05.553973 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:05.572461 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:05.518625 29452 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.007s
I20260709 12:19:05.386606 29251 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.565445 25633 rpcz_store.cc:276] 0709 12:19:04.196604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.551323 (+1354719us) service_pool.cc:224] Handling call
0709 12:19:05.551602 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.551765 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.551788 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.551827 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:05.551854 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.551908 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.552002 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.567080 25784 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
W20260709 12:19:05.570231 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.485232 29511 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:05.555179 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:05.574571 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.439208 29036 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.501309 29517 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.601572 29517 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.584801 25713 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:05.582876 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:05.587356 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82766.8us
W20260709 12:19:05.603536 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.604490 29517 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:05.605793 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:05.606386 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:05.607614 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:05.609005 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:05.610033 25634 raft_consensus.cc:2410] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:19:05.611516 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.591650 25781 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:05.594676 25708 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:05.531142 29507 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:05.470242 29291 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 12:19:05.408041 29357 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:05.602005 25633 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:05.638698 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 804) took 1395 ms. Trace:
W20260709 12:19:05.511684 29334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.006s	sys 0.045s
I20260709 12:19:05.552544 29281 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.470727 29484 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.622854 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.501641 29500 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 12:19:05.675376 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:05.674271 25779 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.011s	sys 0.031s
I20260709 12:19:05.588056 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:05.626863 29512 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:19:05.633279 29315 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.000s
I20260709 12:19:05.465190 29375 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.634646 29436 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:05.635895 29036 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:05.637684 29337 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.106s	user 0.012s	sys 0.001s
I20260709 12:19:05.533843 29365 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:19:05.639041 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 802) took 1438 ms. Trace:
I20260709 12:19:05.703190 25633 rpcz_store.cc:276] 0709 12:19:04.200276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.581845 (+1381569us) service_pool.cc:224] Handling call
0709 12:19:05.601684 (+ 19839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.638824 (+ 37140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.638993 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.639071 25637 rpcz_store.cc:276] 0709 12:19:04.243917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.637943 (+1394026us) service_pool.cc:224] Handling call
0709 12:19:05.638236 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.638392 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.638414 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.638452 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:05.638481 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.638552 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.638648 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:05.640974 29006 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:19:05.642582 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 921) took 1241 ms. Trace:
I20260709 12:19:05.705546 26337 rpcz_store.cc:276] 0709 12:19:04.401031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.401164 (+   133us) service_pool.cc:224] Handling call
0709 12:19:04.417061 (+ 15897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.622728 (+205667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.622774 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.666248 (+ 43474us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.347622 (+681374us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:05.347681 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.347779 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.642338 (+294559us) raft_consensus.cc:1685] finished
0709 12:19:05.642362 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.642523 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43726,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4309,"thread_start_us":843957,"threads_started":2,"wal-append.queue_time_us":952604}
I20260709 12:19:05.645417 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:05.709319 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 461326us
I20260709 12:19:05.709925 25785 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:05.710055 25778 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:05.646019 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.710983 25781 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:05.583981 29499 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.647389 25638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.023s	sys 0.022s
I20260709 12:19:05.624572 29515 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:05.644403 29454 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.711948 29499 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:05.659327 29495 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:05.713142 29495 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:05.713521 29495 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:05.614001 29298 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:05.663969 29294 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:05.714371 29495 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.665508 29251 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:05.666790 29413 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:05.662281 29437 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:05.668545 29513 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:05.669361 29508 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:05.669643 29357 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:05.714461 29499 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:05.679181 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:05.715701 29495 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:05.715911 29437 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:05.677213 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 89249.7us
I20260709 12:19:05.560981 29322 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:05.674192 29323 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:05.652249 29334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.006s	sys 0.046s
I20260709 12:19:05.678627 29042 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:19:05.712446 25638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.023s	sys 0.022s
I20260709 12:19:05.670853 29281 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:05.723287 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:05.676380 25779 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.011s	sys 0.031s
I20260709 12:19:05.726162 26338 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:05.715229 29294 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:05.675235 29517 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:19:05.751053 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 797) took 2002 ms. Trace:
I20260709 12:19:05.702132 29328 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:19:05.679653 29510 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.021s	sys 0.018s
I20260709 12:19:05.733974 29323 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.716044 29513 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.759528 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1831) took 1614 ms. Trace:
I20260709 12:19:05.739604 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:05.727862 29413 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 12:19:05.724551 29337 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.763810 25638 rpcz_store.cc:276] 0709 12:19:03.748565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.701176 (+952611us) service_pool.cc:224] Handling call
0709 12:19:04.701445 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.701659 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:05.107716 (+406057us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.750602 (+642886us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.750679 (+    77us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:05.750736 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.750826 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.750981 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30327,"prepare.queue_time_us":904716,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":18648,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"thread_start_us":1017542,"threads_started":2}
I20260709 12:19:05.728112 29512 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:05.846107 29524 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:19:05.732909 29499 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 12:19:05.744208 29514 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:05.909341 25602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.005s	sys 0.010s
I20260709 12:19:05.910281 29294 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:05.912595 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 189407us
W20260709 12:19:05.952592 25602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.005s	sys 0.010s
I20260709 12:19:05.953137 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:05.913193 29042 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:05.750015 25603 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:05.690932 29315 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.927050 25631 rpcz_store.cc:276] 0709 12:19:04.145019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.886575 (+741556us) service_pool.cc:224] Handling call
0709 12:19:05.448404 (+561829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.468378 (+ 19974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.667234 (+198856us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b540039b820. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5906dc0 00007f29b5992a02 00007f29b598edd6 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:19:05.758064 (+ 90830us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b54001c3ed0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:05.759471 (+  1407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304816256}
I20260709 12:19:05.751811 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 796) took 2015 ms. Trace:
I20260709 12:19:05.956921 25632 rpcz_store.cc:276] 0709 12:19:03.736864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.700360 (+963496us) service_pool.cc:224] Handling call
0709 12:19:04.700778 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.701020 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:05.335128 (+634108us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.751427 (+416299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.751491 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:05.751548 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.751626 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.751749 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28074,"prepare.queue_time_us":798793,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":57,"raft.queue_time_us":18889,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":1018236,"threads_started":2}
I20260709 12:19:05.958242 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:05.958961 25632 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
W20260709 12:19:05.931466 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 191960us
W20260709 12:19:05.948117 25780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.030s	sys 0.011s
W20260709 12:19:05.959733 25780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.030s	sys 0.011s
I20260709 12:19:05.970656 29251 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:19:05.949414 29436 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:05.972487 29436 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.975452 29514 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.991384 29413 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:05.992349 25784 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:05.993531 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1830) took 1848 ms. Trace:
I20260709 12:19:05.993875 25636 rpcz_store.cc:276] 0709 12:19:04.144760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.847956 (+703196us) service_pool.cc:224] Handling call
0709 12:19:04.869705 (+ 21749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.870143 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.870188 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.923694 (+ 53506us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.348435 (+424741us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b18002c6aa0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:19:05.691197 (+342762us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:05.691237 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.691311 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.993198 (+301887us) raft_consensus.cc:1685] finished
0709 12:19:05.993217 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.993462 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16470,"raft.queue_time_us":848,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":43219,"spinlock_wait_cycles":19848960,"thread_start_us":342539,"threads_started":1,"wal-append.queue_time_us":1022234}
I20260709 12:19:05.996760 29391 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:05.997308 29315 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:05.997977 25787 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:05.998713 29512 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:05.999663 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2289) took 1103 ms. Trace:
I20260709 12:19:06.000011 25787 rpcz_store.cc:276] 0709 12:19:04.896163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.896318 (+   155us) service_pool.cc:224] Handling call
0709 12:19:04.896657 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.998353 (+1101696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.999455 (+  1102us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001a86d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:05.999613 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2532433152}
I20260709 12:19:05.944669 29357 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:05.946063 29532 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:06.002938 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:06.003490 26335 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:06.003239 29532 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.005452 29532 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:06.006279 29459 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.494s	user 0.014s	sys 0.000s
I20260709 12:19:06.006671 29459 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.007818 29513 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.008823 29459 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.010064 25631 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:06.010649 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2189) took 1453 ms. Trace:
I20260709 12:19:05.992472 29413 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:05.909552 29006 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:06.011059 25782 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:05.944682 29191 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.011s
W20260709 12:19:06.060741 29191 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.007s	sys 0.011s
I20260709 12:19:06.063269 29375 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.058902 29413 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:06.059367 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 101188us
W20260709 12:19:06.064260 26343 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.023s	sys 0.006s
I20260709 12:19:06.064838 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:06.060176 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 310240us
I20260709 12:19:06.060304 29323 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:06.066202 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 106322us
I20260709 12:19:06.012179 25786 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:06.073942 25635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.001s	sys 0.029s
W20260709 12:19:06.074450 25635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.001s	sys 0.029s
I20260709 12:19:06.076542 26342 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:06.010926 25631 rpcz_store.cc:276] 0709 12:19:04.557334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.009435 (+1452101us) service_pool.cc:224] Handling call
0709 12:19:06.009749 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.010406 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.010595 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.066349 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2188) took 1510 ms. Trace:
I20260709 12:19:06.080109 25632 rpcz_store.cc:276] 0709 12:19:04.556214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.958385 (+1402171us) service_pool.cc:224] Handling call
0709 12:19:05.958741 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:06.066036 (+107295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.066287 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.082943 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:06.083998 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:06.065388 26343 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.023s	sys 0.006s
I20260709 12:19:06.089511 26335 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
W20260709 12:19:06.103238 25709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.024s	sys 0.024s
I20260709 12:19:06.103950 25785 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:19:06.104274 25709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.024s	sys 0.024s
I20260709 12:19:06.094280 29036 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.106216 29261 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:06.107026 29036 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.107923 29261 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:06.072690 29357 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:06.110136 29337 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.110394 29357 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.086050 29429 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:06.112099 29429 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.064184 29283 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.109311 29036 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:06.116034 26341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 806) took 1529 ms (client timeout 1657 ms). Trace:
W20260709 12:19:06.116590 26341 rpcz_store.cc:269] 0709 12:19:04.587129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.587327 (+   198us) service_pool.cc:224] Handling call
0709 12:19:06.115772 (+1528445us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b540058ff54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:06.115960 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646352768}
W20260709 12:19:06.122350 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.014s	sys 0.037s
I20260709 12:19:06.064355 29042 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:06.126262 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.127430 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:06.128175 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:06.085812 29323 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:06.113375 29283 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.134852 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:06.067232 29514 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.126996 25787 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:06.094051 29452 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:06.130359 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.138213 25781 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:06.104564 29532 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:06.144855 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:06.094276 29508 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:06.129616 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.150229 25778 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:06.135977 26341 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:06.121265 29441 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.075680 29391 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:06.175833 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:06.170876 29531 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.257s	user 0.000s	sys 0.002s
I20260709 12:19:06.089433 29006 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.125419 29261 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:06.137679 29452 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:06.176172 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.122721 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.015s	sys 0.037s
W20260709 12:19:06.182873 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 628094us
I20260709 12:19:06.182272 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:06.183925 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:06.171994 29444 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.558s	user 0.006s	sys 0.007s
I20260709 12:19:06.151652 29532 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.197618 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:06.173085 29499 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:06.166932 29508 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.161792 29090 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.172361 29323 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:19:06.183291 29452 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:06.195957 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.196180 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:06.181169 29321 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.157s	user 0.009s	sys 0.004s
I20260709 12:19:06.165534 29536 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.216348 26341 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
W20260709 12:19:06.216711 25711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.016s	sys 0.024s
I20260709 12:19:06.183193 29322 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.188700 29036 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:19:06.223261 29348 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:06.221944 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:06.233446 25711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.017s	sys 0.024s
W20260709 12:19:06.234773 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.001s	sys 0.044s
I20260709 12:19:06.202041 29391 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:06.183383 29484 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.201537 29532 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:06.192039 29511 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:06.188853 29261 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:06.214849 29503 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.016s	sys 0.024s
W20260709 12:19:06.292363 29503 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.017s	sys 0.024s
I20260709 12:19:06.177676 29441 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:06.256378 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.001s	sys 0.044s
I20260709 12:19:06.180106 29538 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 12:19:06.272774 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50921us
I20260709 12:19:06.162003 29413 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:06.243646 26337 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.018s	sys 0.018s
W20260709 12:19:06.300026 26337 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.018s	sys 0.019s
I20260709 12:19:06.237636 29337 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:06.304874 25706 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:19:06.293169 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2449) took 1043 ms. Trace:
I20260709 12:19:06.307226 25711 rpcz_store.cc:276] 0709 12:19:05.249360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.249495 (+   135us) service_pool.cc:224] Handling call
0709 12:19:05.249783 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.250032 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:05.531926 (+281894us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.292742 (+760816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.292800 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.292852 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.292931 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.293098 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27781,"prepare.queue_time_us":538447,"prepare.run_cpu_time_us":7380,"prepare.run_wall_time_us":63565,"thread_start_us":923548,"threads_started":2}
I20260709 12:19:06.311546 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:06.285466 29352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.009s	sys 0.035s
W20260709 12:19:06.300522 25779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.035s	sys 0.042s
W20260709 12:19:06.323541 25779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.035s	sys 0.043s
I20260709 12:19:06.324689 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1805) took 1472 ms. Trace:
I20260709 12:19:06.200043 29444 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:06.296916 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.294345 29507 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.649s	user 0.018s	sys 0.000s
W20260709 12:19:06.295410 29245 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.059s	user 0.011s	sys 0.007s
W20260709 12:19:06.289510 29415 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.811s	user 0.009s	sys 0.006s
I20260709 12:19:06.279968 29459 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:06.336529 25708 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:06.336618 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:06.212340 29499 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:06.328688 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.340364 25629 raft_consensus.cc:2410] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:19:06.273733 29548 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:06.331169 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1804) took 1482 ms. Trace:
W20260709 12:19:06.326355 29352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.009s	sys 0.036s
I20260709 12:19:06.333985 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.342109 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:06.344696 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:06.345235 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:06.359627 25785 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:06.350203 25780 rpcz_store.cc:276] 0709 12:19:04.849073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.849248 (+   175us) service_pool.cc:224] Handling call
0709 12:19:04.879694 (+ 30446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.879957 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:05.960577 (+1080620us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.330401 (+369824us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.330686 (+   285us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.330758 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:06.330827 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.330922 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.331090 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75796,"prepare.queue_time_us":370577,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":130,"raft.queue_time_us":5970,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":20391,"thread_start_us":1299377,"threads_started":2}
I20260709 12:19:06.287374 29511 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.322302 29337 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:06.390746 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 250065us
W20260709 12:19:06.400617 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.401001 25713 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
W20260709 12:19:06.391580 26309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.009s	sys 0.014s
I20260709 12:19:06.251041 29536 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.404978 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:06.407829 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 278943us
I20260709 12:19:06.403923 26340 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:06.407737 25781 raft_consensus.cc:2376] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7. Re-sending same reply.
W20260709 12:19:06.412604 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.408422 26309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.010s	sys 0.014s
W20260709 12:19:06.414669 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 1.05096e+06us
I20260709 12:19:06.332719 29251 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:06.354146 29413 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:06.333235 29090 consensus_queue.cc:1175] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0ae979356d694a09be4b0cd976ea354a" responder_term: 9 status { last_received { term: 9 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:06.348994 29414 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:06.423211 29520 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:19:06.407318 29337 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:06.431644 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.413110 29245 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:06.422358 29321 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:06.433362 29538 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:06.434815 29283 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.435459 29536 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.329602 29315 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:19:06.436209 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 809) took 1081 ms. Trace:
W20260709 12:19:06.436533 29281 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.000s
I20260709 12:19:06.404356 29323 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.438216 29261 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.357985 29294 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.419404 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.401566 29511 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:06.416028 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:06.448843 25785 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:06.323907 29391 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:19:06.412024 29548 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:06.470043 29508 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.338343 29507 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.506466 28983 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:06.472289 29541 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:19:06.474011 29441 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.474779 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.547729 25781 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:06.475395 29444 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.478717 29436 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:06.479259 29543 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:19:06.481810 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.482601 29414 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.415689 29352 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:19:06.483916 29331 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.412335 29036 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:06.488196 29042 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.532095 29281 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.587622 29042 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.488461 29090 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:06.590755 25638 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:06.403709 29545 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:19:06.590253 29281 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.495095 29251 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.490336 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 660) took 1136 ms. Trace:
I20260709 12:19:06.596028 26342 rpcz_store.cc:276] 0709 12:19:05.354034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.354926 (+   892us) service_pool.cc:224] Handling call
0709 12:19:06.490128 (+1135202us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b540044b454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:06.490263 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165287296}
I20260709 12:19:06.503759 29525 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:06.597587 29525 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.349846 29415 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.600225 29415 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:06.464594 29437 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.007s
I20260709 12:19:06.512683 26344 rpcz_store.cc:276] 0709 12:19:05.354375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.354565 (+   190us) service_pool.cc:224] Handling call
0709 12:19:05.355060 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.355550 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:05.355590 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:05.356242 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.927757 (+571515us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:05.927806 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.927907 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.435838 (+507931us) raft_consensus.cc:1685] finished
0709 12:19:06.435865 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.436017 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65256,"raft.queue_time_us":52608,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":39391,"thread_start_us":476408,"threads_started":1,"wal-append.queue_time_us":982940}
I20260709 12:19:06.601027 29437 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.603219 29437 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.529330 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.604508 25637 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:06.537626 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 121819us
I20260709 12:19:06.605199 29499 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.534715 29323 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.463052 29191 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:19:06.506539 29556 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.607518 29323 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.607841 29556 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.610097 29556 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:06.530578 29245 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:06.611594 29352 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:06.612123 29352 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:06.611752 29245 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.612962 29352 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.614019 29352 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:06.615584 25707 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:06.616616 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 808) took 2006 ms. Trace:
I20260709 12:19:06.512629 29538 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.578088 25630 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:06.550637 28983 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.581427 29331 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.581804 29036 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.549480 29006 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.595822 29452 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.503960 29321 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.564237 25782 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:06.619760 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 420489us
I20260709 12:19:06.623092 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.664023 25705 raft_consensus.cc:2410] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:19:06.624125 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:06.590284 29042 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.590936 29090 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:19:06.591069 29551 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:19:06.617015 26343 rpcz_store.cc:276] 0709 12:19:04.610157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.610593 (+   436us) service_pool.cc:224] Handling call
0709 12:19:04.621202 (+ 10609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.360852 (+739650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.360896 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:05.413107 (+ 52211us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.084927 (+671820us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:06.084984 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.085083 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.616368 (+531285us) raft_consensus.cc:1685] finished
0709 12:19:06.616391 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.616550 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104681,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":1339844,"threads_started":2,"wal-append.queue_time_us":855148}
W20260709 12:19:06.634261 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.657s (SENT) [suppressed 24 similar messages]
I20260709 12:19:06.684684 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.650166 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 925) took 1651 ms. Trace:
W20260709 12:19:06.654455 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 356015us
I20260709 12:19:06.563362 29566 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:06.545567 29548 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:06.675649 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.686323 26340 rpcz_store.cc:276] 0709 12:19:04.999194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.999368 (+   174us) service_pool.cc:224] Handling call
0709 12:19:04.999735 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.411509 (+1411774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.649803 (+238294us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400471654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:06.650081 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3220345088}
W20260709 12:19:06.708751 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 360667us
W20260709 12:19:06.716346 25635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.023s
W20260709 12:19:06.716861 25635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.000s	sys 0.023s
I20260709 12:19:06.682976 29090 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:06.729943 26339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.002s	sys 0.022s
W20260709 12:19:06.812559 26339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.005s	sys 0.038s
I20260709 12:19:06.784904 29566 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:06.784375 25709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.010s	sys 0.036s
W20260709 12:19:06.813984 25709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.020s	sys 0.039s
W20260709 12:19:06.811630 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 123904us
I20260709 12:19:06.814270 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 931) took 1168 ms. Trace:
I20260709 12:19:06.706153 25706 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:06.829918 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:06.829967 25705 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:19:06.832710 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:06.832929 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.837651 25705 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:06.814667 26339 rpcz_store.cc:276] 0709 12:19:05.646280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.649360 (+  3080us) service_pool.cc:224] Handling call
0709 12:19:05.691648 (+ 42288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.691899 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:06.134076 (+442177us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.813889 (+679813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.813944 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.813990 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.814091 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.814219 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":600142,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"thread_start_us":1116122,"threads_started":2}
I20260709 12:19:06.845798 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2364) took 2194 ms. Trace:
I20260709 12:19:06.846704 25709 rpcz_store.cc:276] 0709 12:19:04.635508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.635683 (+   175us) service_pool.cc:224] Handling call
0709 12:19:04.636036 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.675260 (+ 39224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.675292 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.675778 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.496877 (+821099us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:06.104791 (+607914us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.829368 (+724577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.829446 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.829489 (+    43us) raft_consensus.cc:1685] finished
0709 12:19:06.829499 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.829618 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63695,"prepare.queue_time_us":451032,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":199585,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":21221,"thread_start_us":2036745,"threads_started":3,"wal-append.queue_time_us":916900}
I20260709 12:19:06.849915 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:06.850970 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.852707 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.853331 25709 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:06.859899 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:06.862406 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:06.842471 29393 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.868619 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.755985 29503 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:06.842724 29495 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:06.881678 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:06.888862 26307 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.536s (SENT)
I20260709 12:19:06.726905 29515 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 12:19:06.859273 29562 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.893591 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.904053 25780 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:06.904994 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.905591 25780 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:06.893810 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:06.846465 29566 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.873973 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:06.894369 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44544.5us
W20260709 12:19:06.907296 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 272970us
I20260709 12:19:06.919112 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.915289 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 803) took 2706 ms. Trace:
I20260709 12:19:06.920223 25781 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:06.916802 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 933) took 1248 ms. Trace:
I20260709 12:19:06.920853 26338 rpcz_store.cc:276] 0709 12:19:05.668893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.669023 (+   130us) service_pool.cc:224] Handling call
0709 12:19:05.669368 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.745743 (+ 76375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.916548 (+1170805us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b54006df454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:06.916739 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126002048}
I20260709 12:19:06.920332 25630 rpcz_store.cc:276] 0709 12:19:04.208394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.603826 (+1395432us) service_pool.cc:224] Handling call
0709 12:19:05.604279 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.619614 (+1015335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.914992 (+295378us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54001d83d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:06.915212 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238888320}
I20260709 12:19:06.921630 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:06.921856 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.906388 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.920361 25709 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:06.923761 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.924450 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:06.924952 25780 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:06.926668 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:06.916971 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 605) took 1405 ms. Trace:
I20260709 12:19:06.930063 25784 rpcz_store.cc:276] 0709 12:19:05.511329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.678966 (+167637us) service_pool.cc:224] Handling call
0709 12:19:06.916713 (+1237747us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400320354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:06.916889 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631394048}
I20260709 12:19:06.931282 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:06.934015 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:06.934394 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:06.935443 25705 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:06.870333 29375 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:06.890872 29415 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.945271 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:06.945231 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.950133 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.951273 25630 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:06.874790 29393 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.955457 29548 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.894075 29503 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:06.961476 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:06.964556 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:06.966097 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.966192 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.968677 25780 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:06.971014 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.972765 25709 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:06.969225 25708 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:06.975915 25603 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:06.897641 29495 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.949452 29499 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:06.944010 29567 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:19:06.982973 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 92928.5us
W20260709 12:19:06.981788 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 123919us
I20260709 12:19:06.956246 29413 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:06.958455 29560 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 12:19:06.950619 29245 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:06.978645 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:06.987392 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:06.991395 25705 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:06.992705 25709 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:19:06.993104 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:06.993791 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:06.892865 29090 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.922183 29562 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:06.956305 29579 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:07.009334 25780 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:07.009334 25630 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:07.242038 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:07.387590 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:07.362356 29281 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:07.243914 29404 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:07.388527 29281 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.299752 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:07.357029 29564 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.641s	user 0.008s	sys 0.000s
I20260709 12:19:07.392547 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:07.340947 29391 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:07.182315 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.116871 29429 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:07.094556 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:07.408186 25787 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:06.325220 25779 rpcz_store.cc:276] 0709 12:19:04.852228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.853198 (+   970us) service_pool.cc:224] Handling call
0709 12:19:04.879110 (+ 25912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.879361 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:05.744158 (+864797us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.324333 (+580175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.324386 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:06.324428 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.324509 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.324631 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":165438,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":18790,"raft.queue_time_us":2011,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":7426,"thread_start_us":1368002,"threads_started":2}
I20260709 12:19:07.145133 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:07.092712 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1832) took 2718 ms. Trace:
I20260709 12:19:07.411109 25634 rpcz_store.cc:276] 0709 12:19:04.374899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.673432 (+1298533us) service_pool.cc:224] Handling call
0709 12:19:05.693094 (+ 19662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.715130 (+ 22036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.715174 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.715231 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:05.715275 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.092430 (+1377155us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b54003016d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:07.092486 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.092643 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":673,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":49818368}
I20260709 12:19:07.207790 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:07.314658 29550 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:07.413885 25630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1848) took 1291 ms (client timeout 1610 ms). Trace:
W20260709 12:19:07.417902 25630 rpcz_store.cc:269] 0709 12:19:06.122732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.964310 (+841578us) service_pool.cc:224] Handling call
0709 12:19:07.413691 (+449381us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b540021f054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:07.413824 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35855232}
I20260709 12:19:07.088088 29331 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:07.088202 29022 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:07.334652 29441 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.304409 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1840) took 2603 ms. Trace:
I20260709 12:19:07.426640 25637 rpcz_store.cc:276] 0709 12:19:04.701602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.010405 (+1308803us) service_pool.cc:224] Handling call
0709 12:19:06.013141 (+  2736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.604949 (+591808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.304127 (+699178us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54001d7ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:07.304339 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1359600896}
W20260709 12:19:07.170164 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 176480us
I20260709 12:19:07.106796 29578 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:19:07.089977 26336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 803) took 2535 ms (client timeout 1667 ms). Trace:
I20260709 12:19:07.371541 29574 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:19:07.298574 29545 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.959312 29375 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.063843 29321 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.297346 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:07.207371 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:07.317456 25534 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:07.111415 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:07.119215 29503 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:06.980705 29538 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.094110 29495 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:07.091766 29337 consensus_meta.cc:287] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.003s
I20260709 12:19:07.251338 29414 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:06.996091 29512 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:06.976869 29525 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:07.101956 29566 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:07.305660 29549 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:06.978044 29510 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:07.123843 29559 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:07.303166 25629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 780) took 4573 ms (client timeout 1838 ms). Trace:
W20260709 12:19:07.537184 25629 rpcz_store.cc:269] 0709 12:19:02.729597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.732367 (+  2770us) service_pool.cc:224] Handling call
0709 12:19:07.209445 (+4477078us) spinlock_profiling.cc:243] Waited 3.6 s on lock 0x7b5400364554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:07.303061 (+ 93616us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8273552640}
I20260709 12:19:07.092362 28983 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:07.301604 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.540066 25708 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:06.995880 29245 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:07.208413 29573 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
W20260709 12:19:07.107196 29516 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.764s	user 0.003s	sys 0.004s
I20260709 12:19:07.126173 29562 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:06.989367 29567 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.296975 25707 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:07.302575 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58873.1us
I20260709 12:19:07.306985 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1843) took 2408 ms. Trace:
I20260709 12:19:07.451959 29441 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.563710 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.124384 29579 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:06.965407 29415 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.567343 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:07.216385 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:07.568486 26339 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:07.223923 29454 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:19:07.304293 29509 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:06.980605 29571 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.076123 29507 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.104511 25633 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:07.582302 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1834) took 3169 ms. Trace:
I20260709 12:19:07.582731 25633 rpcz_store.cc:276] 0709 12:19:04.413405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.732756 (+1319351us) service_pool.cc:224] Handling call
0709 12:19:05.912323 (+179567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.104360 (+1192037us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b540039cc20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5906dc0 00007f29b59911c0 00007f29b5992231 00007f29b598ed8f 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c
0709 12:19:07.582003 (+477643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.582227 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89855360}
I20260709 12:19:07.128810 29036 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.116262 29455 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:07.318698 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:06.982270 29536 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:07.104050 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:07.120134 29348 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:06.981393 29499 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:07.596709 25636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2193) took 1454 ms (client timeout 1624 ms). Trace:
I20260709 12:19:07.572304 29525 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:07.349560 29452 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.362573 29090 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:07.366878 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.604970 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.090271 29393 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.596462 29499 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:07.376649 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.610257 25711 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:07.611146 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.611737 25711 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:07.612661 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.613165 25711 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:07.613894 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.311993 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:07.615445 25781 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:07.616880 25781 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:07.356941 29580 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.269s	user 0.000s	sys 0.005s
I20260709 12:19:07.303423 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:07.619983 29581 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:19:07.337225 29352 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.000s
W20260709 12:19:07.296438 29532 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.998s	user 0.012s	sys 0.002s
I20260709 12:19:07.391734 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.215384 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1839) took 2760 ms. Trace:
I20260709 12:19:07.393410 25780 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:07.398414 25705 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:19:07.298894 26344 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:19:07.634824 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.391367 29281 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.372694 29006 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.431535 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:07.653514 26341 raft_consensus.cc:2410] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:19:07.378834 29322 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:07.402139 29511 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.854s	user 0.012s	sys 0.000s
I20260709 12:19:07.437176 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:07.440703 26336 rpcz_store.cc:269] 0709 12:19:04.555952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.556072 (+   120us) service_pool.cc:224] Handling call
0709 12:19:07.089646 (+2533574us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b54004a4554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:07.089852 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5407420288}
I20260709 12:19:07.657951 26335 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:07.448236 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:07.658867 25630 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
W20260709 12:19:07.659490 25630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1850) took 1536 ms (client timeout 1674 ms). Trace:
W20260709 12:19:07.659891 25630 rpcz_store.cc:269] 0709 12:19:06.123212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.447872 (+1324660us) service_pool.cc:224] Handling call
0709 12:19:07.659428 (+211556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.600811 29348 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:07.474135 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:07.474136 25536 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 194) took 17797 ms (client timeout 15000 ms). Trace:
I20260709 12:19:07.425580 29022 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:07.487349 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 492887us
I20260709 12:19:07.492370 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:07.446949 29515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.005s	sys 0.015s
I20260709 12:19:07.475464 25534 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 6, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 6 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:07.528438 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.648200 29393 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.414196 29391 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.445099 29429 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:19:07.513406 29337 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.541527 25787 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:07.495669 29538 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:07.677994 29538 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.680181 29538 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.681263 25534 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:07.207204 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:07.551787 25631 rpcz_store.cc:276] 0709 12:19:04.898319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.085617 (+1187298us) service_pool.cc:224] Handling call
0709 12:19:06.120978 (+ 35361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.185638 (+ 64660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.185687 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.185749 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:06.185794 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.210588 (+1024794us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b54002ac6d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:07.306784 (+ 96196us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.306924 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148050304}
W20260709 12:19:07.554603 26307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.029s	sys 0.053s
W20260709 12:19:07.684228 26307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.029s	sys 0.053s
W20260709 12:19:07.684602 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 692376us
I20260709 12:19:07.527947 29503 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:07.567047 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:07.569159 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.687141 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.458357 29545 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.439986 29550 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.527861 29549 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:07.579322 25779 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:19:07.580637 25707 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:19:07.539918 29414 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.552723 29245 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:07.584079 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:07.590212 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:07.597167 25636 rpcz_store.cc:269] 0709 12:19:06.141926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.103746 (+961820us) service_pool.cc:224] Handling call
0709 12:19:07.596643 (+492897us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.134517 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:07.563226 29567 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:07.572793 29454 consensus_queue.cc:1243] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:19:07.376581 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.011s
W20260709 12:19:07.708719 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.000s	sys 0.011s
W20260709 12:19:07.709021 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 843635us
I20260709 12:19:07.709028 25784 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:19:07.382647 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:07.622262 29415 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:07.585388 29562 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:07.639003 25705 pending_rounds.cc:85] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 4
I20260709 12:19:07.592541 29536 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.108044 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1810) took 2116 ms. Trace:
I20260709 12:19:07.572516 29495 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:19:07.641010 25638 rpcz_store.cc:276] 0709 12:19:04.455413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.944922 (+1489509us) service_pool.cc:224] Handling call
0709 12:19:05.945383 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.591353 (+645970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.214970 (+623617us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54003007d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:07.215285 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483595136}
I20260709 12:19:07.643121 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:07.637247 25710 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.006s	sys 0.036s
I20260709 12:19:07.638121 25713 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
W20260709 12:19:07.720508 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 337927us
W20260709 12:19:07.626284 29583 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.063s	user 0.001s	sys 0.002s
W20260709 12:19:07.633932 29585 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.261s	user 0.000s	sys 0.002s
W20260709 12:19:07.654417 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 301729us
I20260709 12:19:07.606599 29525 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:07.649469 29452 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.661087 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:07.739538 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:07.739717 25630 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:07.740267 25630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2194) took 1567 ms (client timeout 1947 ms). Trace:
W20260709 12:19:07.740732 25630 rpcz_store.cc:269] 0709 12:19:06.173197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.660745 (+1487548us) service_pool.cc:224] Handling call
0709 12:19:07.740200 (+ 79455us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.459748 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.667732 25637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2192) took 1562 ms (client timeout 1757 ms). Trace:
W20260709 12:19:07.743634 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.743490 25637 rpcz_store.cc:269] 0709 12:19:06.105258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.492012 (+1386754us) service_pool.cc:224] Handling call
0709 12:19:07.667673 (+175661us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.563071 29559 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.475302 29574 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:07.585639 29509 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:07.675261 29515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.005s	sys 0.016s
I20260709 12:19:07.640324 29499 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.701784 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:07.647681 29532 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.587849 29571 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.700366 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:07.662508 29511 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.639537 25711 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:07.725955 25705 pending_rounds.cc:107] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:19:07.820420 25705 raft_consensus.cc:2889] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:07.821310 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2458) took 1111 ms. Trace:
I20260709 12:19:07.821731 25705 rpcz_store.cc:276] 0709 12:19:06.709952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.017174 (+307222us) service_pool.cc:224] Handling call
0709 12:19:07.395235 (+378061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.820986 (+425751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.821242 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.726224 25710 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.007s	sys 0.036s
I20260709 12:19:07.726595 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:07.675535 29579 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.662551 29322 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.738291 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:07.730470 25786 rpcz_store.cc:276] 0709 12:19:04.991588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.991730 (+   142us) service_pool.cc:224] Handling call
0709 12:19:05.023249 (+ 31519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.073557 (+1050308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.069289 (+995732us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b5400086bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:07.107978 (+ 38689us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2273700224}
I20260709 12:19:07.689029 29578 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:07.677242 29352 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:19:07.837393 29495 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:07.704311 29429 consensus_queue.cc:1243] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:07.770040 29281 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.721287 29512 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:19:07.838619 29281 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.734467 29391 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.723762 29548 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.839740 29548 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.831585 29330 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:07.775110 29574 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:07.724578 29593 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.727774 29432 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:07.728093 29536 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:07.785648 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 84004.8us
I20260709 12:19:07.812961 29556 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:07.740306 29525 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:07.687599 29393 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.815637 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 617) took 1469 ms. Trace:
W20260709 12:19:07.825536 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 99061.8us
I20260709 12:19:07.718312 29298 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:19:07.722203 29414 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.837662 29578 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.840658 29281 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:07.657936 29006 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.878162 29432 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:07.880574 29556 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:07.880664 29006 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.742197 29459 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:19:07.735634 29595 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 12:19:07.852985 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:07.881779 29556 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.769484 29452 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.858385 29574 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.779847 29348 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.826819 29090 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.828716 29441 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.812021 29321 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:19:07.905608 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:07.896553 29595 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:07.897193 29432 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:19:07.901404 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48538.2us
I20260709 12:19:07.911523 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:07.896601 29574 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:07.873831 29444 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.871421 29515 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:07.907543 29595 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:07.810976 29549 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:07.904814 29575 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
I20260709 12:19:07.920331 29429 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:07.851188 29593 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.907792 29512 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:07.921101 29455 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:07.913336 29513 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:19:07.924731 29444 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.918253 29515 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:07.933590 29337 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:07.929924 29429 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:07.736663 29562 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:07.969182 29562 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:07.945780 29455 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:07.970146 29455 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:07.966444 29414 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:07.947396 29545 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.959138 29550 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.993238 29559 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.985499 29595 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.008524 29595 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:08.010820 29599 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:19:07.991240 29571 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.012147 29567 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.016917 29567 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:07.663467 25536 rpcz_store.cc:269] 0709 12:18:49.677304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.677868 (+   564us) service_pool.cc:224] Handling call
0709 12:19:07.474052 (+17796184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:01.083653 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:01.134639 (+ 50986us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:01.134692 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:19:01.134711 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:01.136257 (+  1546us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:01.136338 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:01.136371 (+    33us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:19:01.137681 (+  1310us) tablet.cc:701] Row locks acquired
0709 12:19:01.137730 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:19:01.137911 (+   181us) write_op.cc:270] Start()
0709 12:19:01.138070 (+   159us) write_op.cc:276] Timestamp: P: 1783599541137840 usec, L: 0
0709 12:19:01.138092 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:19:01.234855 (+ 96763us) log.cc:844] Serialized 6690 byte log entry
0709 12:19:01.747830 (+512975us) op_driver.cc:464] REPLICATION: finished
0709 12:19:01.830904 (+ 83074us) write_op.cc:301] APPLY: starting
0709 12:19:01.831053 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:01.831562 (+   509us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:01.831788 (+   226us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:01.879429 (+ 47641us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:01.879857 (+   428us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:19:01.879905 (+    48us) write_op.cc:312] APPLY: finished
0709 12:19:01.880357 (+   452us) log.cc:844] Serialized 119 byte log entry
0709 12:19:01.880559 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:01.880742 (+   183us) write_op.cc:454] Released schema lock
0709 12:19:01.880819 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:01.881249 (+   430us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:01.881279 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1307463,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":82696,"apply.run_cpu_time_us":5960,"apply.run_wall_time_us":50364,"num_ops":17,"prepare.queue_time_us":245967,"prepare.run_cpu_time_us":34585,"prepare.run_wall_time_us":351414,"raft.queue_time_us":193773,"raft.run_cpu_time_us":30995,"raft.run_wall_time_us":40051,"replication_time_us":609571,"thread_start_us":359090,"threads_started":4,"wal-append.queue_time_us":259629,"wal-append.run_cpu_time_us":29916,"wal-append.run_wall_time_us":1367855}]]}
I20260709 12:19:07.682747 25534 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:08.053628 25534 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 6, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 6 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:07.748065 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:08.061331 25632 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.024s	sys 0.027s
W20260709 12:19:08.062001 25632 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.024s	sys 0.028s
W20260709 12:19:08.061738 25631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2191) took 1956 ms (client timeout 2179 ms). Trace:
W20260709 12:19:08.062556 25631 rpcz_store.cc:269] 0709 12:19:06.105015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.719587 (+1614572us) service_pool.cc:224] Handling call
0709 12:19:08.061380 (+341793us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:08.063242 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 420163us
I20260709 12:19:08.063742 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1842) took 3237 ms. Trace:
I20260709 12:19:08.064867 25603 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.066025 25633 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:07.735600 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:07.689565 25629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1852) took 1565 ms (client timeout 1740 ms). Trace:
I20260709 12:19:08.067386 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2383) took 1764 ms. Trace:
I20260709 12:19:07.741918 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.067741 25638 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
W20260709 12:19:08.068866 25638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1856) took 1476 ms (client timeout 1702 ms). Trace:
I20260709 12:19:08.069386 25630 raft_consensus.cc:2410] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:19:08.069854 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2451) took 1375 ms. Trace:
I20260709 12:19:07.728615 25634 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:08.067920 25704 rpcz_store.cc:276] 0709 12:19:06.302977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.303959 (+   982us) service_pool.cc:224] Handling call
0709 12:19:06.335871 (+ 31912us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:06.414131 (+ 78260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.414273 (+   142us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:06.436370 (+ 22097us) log.cc:844] Serialized 52 byte log entry
0709 12:19:06.754310 (+317940us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:19:06.754355 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.754422 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:08.066523 (+1312101us) raft_consensus.cc:1685] finished
0709 12:19:08.066574 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.067021 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117498,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":20073,"thread_start_us":317438,"threads_started":1,"wal-append.queue_time_us":1137822}
W20260709 12:19:07.732915 25635 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.023s	sys 0.020s
I20260709 12:19:08.071321 25706 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:19:08.071923 25635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.354s	user 0.024s	sys 0.022s
I20260709 12:19:07.808588 25707 raft_consensus.cc:2410] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:08.070366 25709 rpcz_store.cc:276] 0709 12:19:06.705814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.002200 (+296386us) service_pool.cc:224] Handling call
0709 12:19:07.002920 (+   720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.003452 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.003491 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:07.004478 (+   987us) log.cc:844] Serialized 26 byte log entry
0709 12:19:07.649518 (+645040us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:07.649566 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.649641 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:08.069207 (+419566us) raft_consensus.cc:1685] finished
0709 12:19:08.069266 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.069617 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48065,"raft.queue_time_us":138,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":64116,"thread_start_us":278708,"threads_started":1,"wal-append.queue_time_us":718131}
I20260709 12:19:07.744510 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:08.073134 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.072454 25704 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:08.074401 25637 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:08.074570 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 556) took 1468 ms. Trace:
I20260709 12:19:08.075134 25712 rpcz_store.cc:276] 0709 12:19:06.606295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.608667 (+  2372us) service_pool.cc:224] Handling call
0709 12:19:06.609112 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:06.609524 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.609561 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:06.610164 (+   603us) log.cc:844] Serialized 52 byte log entry
0709 12:19:06.891358 (+281194us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:06.891394 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.891470 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.625516 (+734046us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:08.073657 (+448141us) raft_consensus.cc:1685] finished
0709 12:19:08.073721 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.074295 (+   574us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5756,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":18618624,"thread_start_us":280794,"threads_started":1,"wal-append.queue_time_us":846747}
I20260709 12:19:07.813040 25677 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:08.074586 25631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1854) took 1590 ms (client timeout 1987 ms). Trace:
W20260709 12:19:08.087025 25631 rpcz_store.cc:269] 0709 12:19:06.484057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.064484 (+1580427us) service_pool.cc:224] Handling call
0709 12:19:08.074469 (+  9985us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:08.087787 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 274776us
I20260709 12:19:08.088864 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.089973 25631 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:08.090617 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1860) took 1134 ms. Trace:
I20260709 12:19:08.091154 25631 rpcz_store.cc:276] 0709 12:19:06.955868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.088218 (+1132350us) service_pool.cc:224] Handling call
0709 12:19:08.090520 (+  2302us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.831806 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 93602.8us
I20260709 12:19:08.092818 25711 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:19:08.077171 25534 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 4, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:19:08.067847 25629 rpcz_store.cc:269] 0709 12:19:06.123703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.538040 (+1414337us) service_pool.cc:224] Handling call
0709 12:19:07.689490 (+151450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.096496 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.095417 25534 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 3 to 5, term changed from 3 to 6, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to <none>, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 6 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:08.097728 25629 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:08.098435 25534 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 7, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 7 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:08.092783 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.101727 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:08.071231 25634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1845) took 1949 ms (client timeout 1652 ms). Trace:
W20260709 12:19:08.108752 25634 rpcz_store.cc:269] 0709 12:19:06.121927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.446356 (+1324429us) service_pool.cc:224] Handling call
0709 12:19:08.070990 (+624634us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:08.103276 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 355176us
I20260709 12:19:08.110963 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.111267 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:08.111868 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.112320 25634 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:08.113013 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2190) took 2010 ms. Trace:
W20260709 12:19:08.070130 25630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1847) took 1947 ms (client timeout 2243 ms). Trace:
I20260709 12:19:08.113467 25634 rpcz_store.cc:276] 0709 12:19:06.104694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.110310 (+2005616us) service_pool.cc:224] Handling call
0709 12:19:08.112941 (+  2631us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:08.113924 25630 rpcz_store.cc:269] 0709 12:19:06.122453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.741596 (+1619143us) service_pool.cc:224] Handling call
0709 12:19:08.070040 (+328444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.843209 25782 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:19:07.861892 25785 rpcz_store.cc:276] 0709 12:19:06.345822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.538910 (+193088us) service_pool.cc:224] Handling call
0709 12:19:06.539799 (+   889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.540323 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:06.575368 (+ 35045us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.989365 (+413997us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.989862 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.989991 (+   129us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:06.990215 (+   224us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.990369 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.990439 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.815306 (+824867us) raft_consensus.cc:1685] finished
0709 12:19:07.815340 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.815549 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":414,"prepare.queue_time_us":38909,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":109,"raft.queue_time_us":342091,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":5139,"thread_start_us":413549,"threads_started":2,"wal-append.queue_time_us":875757}
W20260709 12:19:08.107721 25778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.034s	sys 0.038s
W20260709 12:19:08.119925 25778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.034s	sys 0.038s
I20260709 12:19:08.121420 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2308) took 1171 ms. Trace:
I20260709 12:19:07.915727 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:08.124037 28983 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:08.124567 29509 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:08.125185 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 209405us
I20260709 12:19:08.126962 28983 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:19:08.128131 29322 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:07.908855 29090 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.130255 29511 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:08.131819 29393 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.135191 29515 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:07.961086 29429 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:07.987069 29532 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:08.069427 25638 rpcz_store.cc:269] 0709 12:19:06.592879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.735331 (+1142452us) service_pool.cc:224] Handling call
0709 12:19:08.068758 (+333427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.128263 29509 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:08.119594 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.109117 25631 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:08.119140 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.139214 29414 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:08.147511 29414 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.148038 29414 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:08.150840 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.141974 29578 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:08.152359 29578 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.152727 29578 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:08.140163 29562 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:08.158085 29562 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.158527 29562 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:08.160903 25633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1849) took 2037 ms (client timeout 1961 ms). Trace:
W20260709 12:19:08.161413 25633 rpcz_store.cc:269] 0709 12:19:06.122990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.583800 (+1460810us) service_pool.cc:224] Handling call
0709 12:19:08.160550 (+576750us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54002abcd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:08.160820 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822722304}
I20260709 12:19:08.162932 29532 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:08.163347 29532 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:08.163765 29532 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:08.164487 25634 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:08.165102 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1853) took 1681 ms. Trace:
I20260709 12:19:08.165416 25634 rpcz_store.cc:276] 0709 12:19:06.483725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.118659 (+1634934us) service_pool.cc:224] Handling call
0709 12:19:08.165011 (+ 46352us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:08.166199 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.168109 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2300) took 1296 ms. Trace:
I20260709 12:19:08.168484 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2299) took 1320 ms. Trace:
I20260709 12:19:08.169000 25782 rpcz_store.cc:276] 0709 12:19:06.848169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.848343 (+   174us) service_pool.cc:224] Handling call
0709 12:19:06.848778 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.119189 (+1270411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.167963 (+ 48774us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54001252d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:08.168327 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285794688}
I20260709 12:19:08.172667 25534 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: leader changed from <none> to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 4 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:08.175325 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:08.113579 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:08.064281 25632 rpcz_store.cc:276] 0709 12:19:04.826163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.081172 (+1255009us) service_pool.cc:224] Handling call
0709 12:19:06.081674 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.549858 (+468184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.549909 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.553402 (+  3493us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.927557 (+374155us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:07.416897 (+489340us) log.cc:844] Serialized 14 byte log entry
0709 12:19:08.062896 (+645999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.063129 (+   233us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:08.063278 (+   149us) raft_consensus.cc:1685] finished
0709 12:19:08.063313 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.063587 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21362,"prepare.queue_time_us":739866,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":119810,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":18560,"spinlock_wait_cycles":27520,"thread_start_us":1947831,"threads_started":4,"wal-append.queue_time_us":534425}
I20260709 12:19:08.122617 25603 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.180768 29375 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:08.208184 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1859) took 1208 ms. Trace:
I20260709 12:19:08.187301 29545 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:08.187983 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.188241 29484 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:08.190514 29444 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:08.191232 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.121837 25778 rpcz_store.cc:276] 0709 12:19:06.949589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.512801 (+563212us) service_pool.cc:224] Handling call
0709 12:19:07.513183 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.527138 (+ 13955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.527177 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:07.537232 (+ 10055us) log.cc:844] Serialized 26 byte log entry
0709 12:19:08.120713 (+583481us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:08.120816 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.120940 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:08.121016 (+    76us) raft_consensus.cc:1685] finished
0709 12:19:08.121028 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.121258 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13578,"raft.queue_time_us":181,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":18473,"thread_start_us":538718,"threads_started":1,"wal-append.queue_time_us":410365}
I20260709 12:19:08.193971 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:08.195034 29499 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.195294 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:08.197150 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:08.201804 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:08.206037 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.212750 25631 rpcz_store.cc:276] 0709 12:19:06.936162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.092185 (+1156023us) service_pool.cc:224] Handling call
0709 12:19:08.144833 (+ 52648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.168772 25784 rpcz_store.cc:276] 0709 12:19:06.871151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.932106 (+ 60955us) service_pool.cc:224] Handling call
0709 12:19:06.932385 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.709726 (+777341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.167238 (+457512us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400098654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:08.167691 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785135488}
I20260709 12:19:08.218566 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.217516 29484 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:08.138424 29515 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:08.235175 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 112802us
I20260709 12:19:08.237957 25638 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:08.215690 29545 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.262051 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.142622 29579 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 12:19:08.254169 29545 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:08.255323 25534 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 1 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 4 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:08.279105 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 190018us
I20260709 12:19:08.283631 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:08.281453 25630 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:08.201745 29432 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:19:08.280557 29393 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:08.297025 29591 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:19:08.299295 25713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.024s	sys 0.019s
I20260709 12:19:08.299190 29452 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.299803 29567 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:08.300287 29589 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:08.297575 25634 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:08.313412 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.314980 29571 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:08.315189 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.315675 29571 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:08.319232 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44925us
I20260709 12:19:08.304569 29393 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.326138 29393 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:08.318739 25534 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 5 to 6, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 6 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:08.325381 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:08.335829 29573 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.765s	user 0.007s	sys 0.000s
I20260709 12:19:08.338052 29508 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:19:08.338896 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 225618us
I20260709 12:19:08.339923 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:08.341022 29499 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:08.341739 29593 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:08.345286 29549 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:08.347049 29484 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.332222 25534 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:08.349900 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.322367 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:08.355610 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.356379 25704 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:08.357246 29337 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:08.360503 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.381879 25637 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
W20260709 12:19:08.387123 25637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1857) took 1741 ms (client timeout 1816 ms). Trace:
W20260709 12:19:08.387642 25637 rpcz_store.cc:269] 0709 12:19:06.645369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.744273 (+1098904us) service_pool.cc:224] Handling call
0709 12:19:08.382519 (+638246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.341748 29499 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:08.386727 25534 catalog_manager.cc:5697] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:08.398083 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:08.397744 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.401101 25782 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:08.401216 25534 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:08.402100 29602 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.402822 25534 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 4, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:08.404896 25534 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 5 to 6, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 6 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:08.406733 29568 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:08.407533 29568 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:08.410980 29525 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.005s
I20260709 12:19:08.381403 29591 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.411674 29525 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.411890 29591 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.413977 29525 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:08.406440 25534 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:08.410908 25630 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:08.424023 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.423331 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 810) took 2818 ms. Trace:
I20260709 12:19:08.416671 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 574) took 1020 ms. Trace:
I20260709 12:19:08.437618 25637 rpcz_store.cc:276] 0709 12:19:05.605140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.406561 (+2801421us) service_pool.cc:224] Handling call
0709 12:19:08.407028 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.407382 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.407411 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:08.423008 (+ 15597us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:08.423050 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.423119 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.423243 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.428767 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.424624 25638 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:08.436764 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 799) took 4203 ms. Trace:
I20260709 12:19:08.418247 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:08.438555 25713 rpcz_store.cc:276] 0709 12:19:07.395863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.730856 (+334993us) service_pool.cc:224] Handling call
0709 12:19:07.731315 (+   459us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:07.731746 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.731775 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:07.732526 (+   751us) log.cc:844] Serialized 78 byte log entry
0709 12:19:08.326851 (+594325us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:08.326941 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.327145 (+   204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:08.416374 (+ 89229us) raft_consensus.cc:1685] finished
0709 12:19:08.416392 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.416577 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2184,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":160,"thread_start_us":576998,"threads_started":1,"wal-append.queue_time_us":544191}
I20260709 12:19:08.430555 25534 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:08.449406 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 64694.2us
W20260709 12:19:08.446053 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 140157us
I20260709 12:19:08.428370 29509 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:19:08.434208 29503 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.725s	user 0.006s	sys 0.006s
W20260709 12:19:08.455987 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71097us
W20260709 12:19:08.418071 29245 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.007s
I20260709 12:19:08.463414 29484 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:08.464028 29589 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:08.463768 29441 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:08.464484 29589 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.464968 29589 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:08.466166 29487 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.414407 29591 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:08.458936 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 71499.3us
W20260709 12:19:08.295445 29352 consensus_queue.cc:1175] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0ae979356d694a09be4b0cd976ea354a" responder_term: 7 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:08.469571 25634 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:08.457505 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.428567 29321 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.472769 29429 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.475019 29512 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:08.477784 29579 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:08.478555 29613 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:19:08.479172 29315 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:08.479777 29602 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.463824 29245 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.466852 29487 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.461701 25534 catalog_manager.cc:5697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:08.488440 29512 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.500205 29512 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:08.500505 29613 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:08.501041 29613 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:08.439917 25635 rpcz_store.cc:276] 0709 12:19:04.192379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.882647 (+690268us) service_pool.cc:224] Handling call
0709 12:19:04.882963 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.883185 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:05.364330 (+481145us) log.cc:844] Serialized 14 byte log entry
0709 12:19:08.072705 (+2708375us) log.cc:844] Serialized 14 byte log entry
0709 12:19:08.321006 (+248301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:08.368038 (+ 47032us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:08.375441 (+  7403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.385016 (+  9575us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.395209 (+ 10193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41426,"prepare.queue_time_us":1653976,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":2449,"raft.queue_time_us":59267,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":226,"thread_start_us":3364577,"threads_started":5,"wal-append.queue_time_us":962657,"wal-append.run_cpu_time_us":1869,"wal-append.run_wall_time_us":1240748}
I20260709 12:19:08.506982 25779 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:08.508112 25635 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:08.509027 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1846) took 2386 ms. Trace:
I20260709 12:19:08.509649 25786 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:19:08.509579 25635 rpcz_store.cc:276] 0709 12:19:06.122215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.507054 (+2384839us) service_pool.cc:224] Handling call
0709 12:19:08.507523 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.508630 (+  1107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.508945 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.510198 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2468) took 1522 ms. Trace:
I20260709 12:19:08.510524 29602 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.510730 25706 rpcz_store.cc:276] 0709 12:19:06.987657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.125645 (+137988us) service_pool.cc:224] Handling call
0709 12:19:07.369141 (+243496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.072179 (+703038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.509849 (+437670us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54003ab454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:08.510130 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1613110528}
I20260709 12:19:08.511363 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.512210 25787 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:08.513203 29571 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.514583 29499 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.516433 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.517454 29245 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.493288 29579 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:08.520123 29579 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.520985 25629 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:08.521714 29579 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:08.495956 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:08.446831 29562 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.536566 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:08.486179 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 542298us
I20260709 12:19:08.482089 29567 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 12:19:08.555752 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 85202.3us
I20260709 12:19:08.495028 29315 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:08.548094 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:08.548980 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:08.550427 29510 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.013s	user 0.012s	sys 0.008s
I20260709 12:19:08.550905 29429 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.551378 25630 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:08.535997 25710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.017s	sys 0.018s
I20260709 12:19:08.462455 29414 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.573647 25782 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:08.568624 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 93965.6us
I20260709 12:19:08.537668 25534 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 6, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 6 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:08.579115 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.486759 29578 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.488219 29321 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.502164 29613 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:08.598285 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 63797.5us
I20260709 12:19:08.612717 25705 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:08.617838 25636 raft_consensus.cc:2368] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:08.616623 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.621840 25711 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:08.622118 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.623172 25787 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:08.628062 29090 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.629966 29322 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:08.630442 29322 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:08.630813 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 134950us
I20260709 12:19:08.631114 29548 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.631808 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:08.632376 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.633261 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.631492 29322 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.638206 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:08.638609 29594 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:08.639078 25779 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:08.639287 29594 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.623116 25706 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:08.594614 29510 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:08.650441 25713 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:08.643175 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:08.615020 29503 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:08.653841 25629 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
W20260709 12:19:08.657466 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 163855us
I20260709 12:19:08.659920 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1863) took 1289 ms. Trace:
W20260709 12:19:08.660593 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.660028 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1855) took 2138 ms. Trace:
I20260709 12:19:08.660307 25629 rpcz_store.cc:276] 0709 12:19:07.370350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.538425 (+1168075us) service_pool.cc:224] Handling call
0709 12:19:08.608676 (+ 70251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.658971 (+ 50295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.659201 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.662732 25635 rpcz_store.cc:276] 0709 12:19:06.516388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.511367 (+1994979us) service_pool.cc:224] Handling call
0709 12:19:08.512308 (+   941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.512538 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:08.540616 (+ 28078us) log.cc:844] Serialized 14 byte log entry
0709 12:19:08.646019 (+105403us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:08.646640 (+   621us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:08.654296 (+  7656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.654383 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.654548 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33229,"thread_start_us":98363,"threads_started":2}
I20260709 12:19:08.663326 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:08.674580 29614 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:08.676440 29444 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:08.677065 29444 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.658804 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:08.683171 29404 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.290s	user 0.017s	sys 0.005s
I20260709 12:19:08.683485 25711 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:19:08.684295 29513 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:08.684751 29513 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.685169 29513 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:08.685277 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 814) took 1040 ms. Trace:
I20260709 12:19:08.685586 29022 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.685793 25638 rpcz_store.cc:276] 0709 12:19:07.644460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.662867 (+1018407us) service_pool.cc:224] Handling call
0709 12:19:08.684695 (+ 21828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.684900 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.684931 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:08.684996 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:08.685027 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.685085 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.685220 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.653638 29429 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:08.694880 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.695395 25712 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:19:08.690649 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:08.688663 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 557) took 2068 ms. Trace:
I20260709 12:19:08.662554 25534 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 7, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 7 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:08.687013 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.713749 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:08.683528 29455 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.713s	user 0.010s	sys 0.007s
I20260709 12:19:08.703459 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.716753 25786 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:08.630463 29512 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.692668 29444 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:08.687510 29536 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.815s	user 0.013s	sys 0.004s
I20260709 12:19:08.722573 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:08.711673 29562 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.722615 29571 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:08.723107 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.724709 29022 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:08.725121 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 70721.7us
I20260709 12:19:08.755760 25534 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 7 to 6, leader changed from <none> to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 6 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:19:08.711957 29532 consensus_meta.cc:287] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.548s	user 0.002s	sys 0.012s
I20260709 12:19:08.688951 29414 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.706104 29429 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:08.772231 29429 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:08.756152 29571 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:08.777367 29571 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:08.779422 29487 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:08.755475 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 109003us
W20260709 12:19:08.783962 29612 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:08.784205 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:08.785640 29624 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:19:08.785866 29536 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.787871 29321 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.788390 29536 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.788538 29321 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:08.788993 29321 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.789265 25780 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:19:08.789369 29321 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:08.790403 29594 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:08.793694 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.795012 25631 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:08.667676 29589 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.762111 25534 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 3, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 3 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:08.706888 25710 rpcz_store.cc:276] 0709 12:19:06.620345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.622568 (+  2223us) service_pool.cc:224] Handling call
0709 12:19:06.659219 (+ 36651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.659522 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:07.825019 (+1165497us) log.cc:844] Serialized 14 byte log entry
0709 12:19:08.620507 (+795488us) log.cc:844] Serialized 14 byte log entry
0709 12:19:08.653655 (+ 33148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:08.660337 (+  6682us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:08.666764 (+  6427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.688405 (+ 21641us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.688594 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72008,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":22862,"raft.queue_time_us":9204,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":22988,"thread_start_us":1880077,"threads_started":2,"wal-append.queue_time_us":593529}
I20260709 12:19:08.763397 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.802662 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.803481 25706 raft_consensus.cc:2410] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:19:08.758515 29322 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:08.803671 26342 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:08.755810 29455 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.805744 26336 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:08.806521 29455 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:08.808135 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1858) took 2151 ms. Trace:
I20260709 12:19:08.808516 25631 rpcz_store.cc:276] 0709 12:19:06.657125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.359954 (+1702829us) service_pool.cc:224] Handling call
0709 12:19:08.807924 (+447970us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54001e91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:08.808074 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":996910976}
I20260709 12:19:08.809175 26341 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:08.810738 25633 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:08.812515 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 809) took 3475 ms. Trace:
I20260709 12:19:08.812839 25633 rpcz_store.cc:276] 0709 12:19:05.337626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.442843 (+3105217us) service_pool.cc:224] Handling call
0709 12:19:08.810574 (+367731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.811110 (+   536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.812304 (+  1194us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54001e91d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:08.812430 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803889536}
W20260709 12:19:08.813926 29515 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.005s
I20260709 12:19:08.763734 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.763942 29548 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:08.764485 25638 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:08.768977 29595 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:08.769490 29614 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:08.771557 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:08.716846 29613 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:08.785666 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.807873 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 821) took 1117 ms. Trace:
I20260709 12:19:08.817883 25534 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:08.753983 29525 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:08.823678 25779 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:08.815915 25633 pending_rounds.cc:85] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 6
I20260709 12:19:08.820262 29594 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:08.831311 28983 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:08.832159 28983 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:08.832381 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60949.2us
I20260709 12:19:08.833285 28983 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.831426 29594 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.836817 25780 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: [4.6-7.7]
I20260709 12:19:08.821909 25636 rpcz_store.cc:276] 0709 12:19:07.690859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.806914 (+1116055us) service_pool.cc:224] Handling call
0709 12:19:08.807323 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.807536 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.807560 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:08.807613 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:08.807643 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.807706 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.807801 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.827446 25633 pending_rounds.cc:107] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:19:08.814575 29578 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.849406 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.849610 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:08.850929 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 73022.8us
I20260709 12:19:08.851558 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:08.854717 26338 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:08.854894 29619 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:08.855688 29414 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:08.820541 29548 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:08.846247 25633 raft_consensus.cc:2889] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.856868 25779 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:08.858091 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:08.863544 25635 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:19:08.865847 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 80339.9us
I20260709 12:19:08.754812 29545 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.844393 29594 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:08.872083 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 819) took 1218 ms. Trace:
I20260709 12:19:08.873821 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.815554 29487 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:19:08.875721 25635 rpcz_store.cc:276] 0709 12:19:07.654101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.753402 (+1099301us) service_pool.cc:224] Handling call
0709 12:19:08.847042 (+ 93640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.863950 (+ 16908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.872008 (+  8058us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:08.800113 29532 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.880834 29090 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:08.883191 25783 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:08.890592 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:08.891764 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 812) took 3226 ms. Trace:
I20260709 12:19:08.864323 29548 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:08.856029 29619 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:08.902580 25630 rpcz_store.cc:276] 0709 12:19:05.665404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.455770 (+2790366us) service_pool.cc:224] Handling call
0709 12:19:08.488597 (+ 32827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.577408 (+ 88811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.891493 (+314085us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b5400301bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:08.891698 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143931776}
I20260709 12:19:08.871711 25534 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 2 to 3, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:08.847859 29536 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:19:08.894340 29503 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:08.907810 29348 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.907539 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:08.926945 29545 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.927613 25637 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:08.901568 29090 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:08.954044 25603 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:08.868257 29455 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:08.907294 29589 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:08.845714 29510 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:19:08.974896 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.974964 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.976468 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:08.873935 29414 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:08.947754 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:08.856514 29245 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:19:08.977947 25704 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:08.848867 29567 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:19:08.978400 29414 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:08.979323 29567 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:08.979933 29567 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:08.980752 29532 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:08.953685 29348 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.959213 29513 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.962929 29455 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:08.962672 29593 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:08.963167 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:08.963943 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.964071 26338 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:08.964846 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.000839 26336 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:08.999185 25710 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:08.982730 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50648us
I20260709 12:19:08.932168 29619 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:08.965147 29549 consensus_meta.cc:287] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.619s	user 0.000s	sys 0.012s
I20260709 12:19:08.871078 29614 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:19:08.965868 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.966082 29571 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:08.966719 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.966990 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.968348 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:08.969358 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.970180 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.943176 25534 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 4, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:08.971539 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:08.972998 29429 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:08.989642 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 136335us
I20260709 12:19:08.899895 29578 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:08.991223 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43546.7us
I20260709 12:19:09.017271 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:08.998720 29455 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:08.976341 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.022002 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.027074 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:09.027304 25712 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:09.028170 26340 raft_consensus.cc:2393] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:09.028959 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.004742 29245 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:09.030421 29245 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:09.030896 29245 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:09.030961 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:08.982054 29090 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:09.034929 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.035923 25781 raft_consensus.cc:2410] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:19:09.036903 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.038105 29331 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:08.980934 29567 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.038727 29331 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.039280 29567 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:09.040969 29331 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.024179 29578 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:09.035861 29628 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:19:09.042410 26339 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:09.042322 29602 consensus_queue.cc:1175] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7bf8f20701245bcbaa09cc663025f97" responder_term: 11 status { last_received { term: 9 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.7-10.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:09.044801 29429 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:09.042119 29444 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:09.045830 29444 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:09.046758 29532 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:09.047525 25711 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:09.050347 29619 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.023597 29513 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:09.034929 26341 raft_consensus.cc:2368] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:09.051604 29619 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:09.052126 29545 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:09.050817 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:09.045874 25676 leader_election.cc:336] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.610s (SENT)
I20260709 12:19:09.055508 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:09.056787 29571 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:09.057099 25676 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.657s (SENT)
I20260709 12:19:09.057410 25712 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:09.057525 25629 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:09.058032 29322 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:19:09.060583 29509 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:09.061200 29509 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:09.063230 29568 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.064922 26339 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:09.060269 29090 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.068565 29578 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:09.030000 26343 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:09.071038 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:09.072569 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:09.054975 29512 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:09.075913 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.081188 25711 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:09.082126 29525 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.055008 25709 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:19:09.082690 29525 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.062274 29509 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.082453 29548 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.085721 29525 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:09.086419 29525 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:09.073079 29090 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.088977 29090 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:09.089718 29090 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:09.091348 29090 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.052039 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.092492 29315 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:09.083626 29393 consensus_meta.cc:287] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.000s
I20260709 12:19:09.094096 29315 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:09.091890 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.097632 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.098814 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.099574 25787 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:09.100314 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.100868 25787 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:09.046949 29444 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.095398 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 820) took 1440 ms. Trace:
I20260709 12:19:09.082090 25712 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:09.102082 25637 rpcz_store.cc:276] 0709 12:19:07.655446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.782454 (+1127008us) service_pool.cc:224] Handling call
0709 12:19:08.782945 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.951187 (+168242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.095157 (+143970us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400363b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:09.095338 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332119808}
W20260709 12:19:09.007535 25601 connection.cc:256] RPC call timeout handler was delayed by -1.1618577689998801s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 12:19:09.104676 25634 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.001s
I20260709 12:19:09.105110 25634 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
W20260709 12:19:09.095561 25676 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.667s (SENT)
I20260709 12:19:09.105849 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 817) took 1460 ms. Trace:
I20260709 12:19:09.094244 29393 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.080163 25710 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:09.106295 25634 rpcz_store.cc:276] 0709 12:19:07.645189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.351195 (+706006us) service_pool.cc:224] Handling call
0709 12:19:09.105772 (+754577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:09.106407 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:09.047619 29532 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.109411 25704 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:09.100594 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.123272 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.124396 25780 raft_consensus.cc:2410] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:19:09.126451 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:09.127305 25704 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:09.127475 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.128991 25710 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:09.124780 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:09.169354 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.131824 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.131044 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.174644 25634 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:09.134060 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.176939 25704 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:19:09.134384 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.178237 25630 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:09.121198 29532 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:09.186312 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 243929us
I20260709 12:19:09.189576 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:09.186808 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 55472.5us
I20260709 12:19:09.190362 25635 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:09.108395 29331 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:09.202704 25712 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:09.204873 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:09.206408 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 81715.3us
I20260709 12:19:09.207262 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:09.218225 25632 raft_consensus.cc:2368] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10.
W20260709 12:19:09.226584 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 601) took 3694 ms (client timeout 1728 ms). Trace:
I20260709 12:19:09.229938 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.234705 25637 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:09.227258 25783 rpcz_store.cc:269] 0709 12:19:05.481733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.514945 (+ 33212us) service_pool.cc:224] Handling call
0709 12:19:09.131111 (+3616166us) spinlock_profiling.cc:243] Waited 3.17 s on lock 0x7b54001a77d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:09.175763 (+ 44652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7285122304}
I20260709 12:19:09.230842 25709 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:09.255300 25787 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:09.257376 25704 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:09.258371 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.262785 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:09.275921 29635 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.088s	user 0.004s	sys 0.000s
I20260709 12:19:09.277828 29548 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.177098 29525 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:09.277868 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.278841 29525 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.279881 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.280663 25784 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:09.281252 29525 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.281414 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.286790 29512 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:09.290318 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.290869 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.291492 25784 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:09.300015 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 92839.9us
I20260709 12:19:09.262874 25707 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:19:09.305430 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:09.306233 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.258814 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:09.309558 25710 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:09.311569 25712 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:09.312471 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.303954 25780 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:19:09.203402 29331 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.338517 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.342051 25638 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:09.343066 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.133049 29571 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.345429 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 816) took 1675 ms. Trace:
I20260709 12:19:09.350466 25629 rpcz_store.cc:276] 0709 12:19:07.644950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.662198 (+1017248us) service_pool.cc:224] Handling call
0709 12:19:08.753604 (+ 91406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.058168 (+304564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.318269 (+260101us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400362c54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:09.320807 (+  2538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698376960}
I20260709 12:19:09.351442 25786 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:09.338989 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:09.355718 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.359048 25634 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:09.359591 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:09.360129 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.362093 25713 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:19:09.362568 25713 pending_rounds.cc:85] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 8
I20260709 12:19:09.362756 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.362802 25713 pending_rounds.cc:107] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:19:09.363909 25713 raft_consensus.cc:2889] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:09.364357 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:09.368382 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.364521 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:09.363674 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.253010 29595 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:19:09.348340 29331 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:09.348513 29602 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:19:09.357146 29511 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.375444 25637 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:09.387156 25783 raft_consensus.cc:2410] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:19:09.387478 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.388312 25785 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:19:09.391213 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.348097 29614 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
W20260709 12:19:09.430047 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.384852 25784 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:09.389801 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 827) took 1015 ms. Trace:
I20260709 12:19:09.419836 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:09.369796 25786 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:09.380573 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.380162 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:09.417416 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55724.9us
I20260709 12:19:09.423221 25603 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:09.442849 26341 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:09.412297 25781 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:09.380380 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.454245 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.455897 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.447319 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.440814 25633 rpcz_store.cc:276] 0709 12:19:08.374397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.813820 (+439423us) service_pool.cc:224] Handling call
0709 12:19:08.814165 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:08.956945 (+142780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.956998 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:08.957649 (+   651us) log.cc:844] Serialized 52 byte log entry
0709 12:19:09.113782 (+156133us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:09.113823 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.113896 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:09.389583 (+275687us) raft_consensus.cc:1685] finished
0709 12:19:09.389606 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:09.389740 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5870,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":155669,"threads_started":1,"wal-append.queue_time_us":429840}
W20260709 12:19:09.402576 29321 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.006s
I20260709 12:19:09.457698 26339 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:09.373996 29545 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:09.379874 29550 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:09.465975 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:09.458923 29594 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
I20260709 12:19:09.383095 29495 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:19:09.432412 29613 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.003s
I20260709 12:19:09.470432 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.372691 29375 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.380028 29641 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:09.472994 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.501482 25786 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 12:19:09.360395 29571 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.477984 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:09.373317 29414 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:19:09.480229 25784 raft_consensus.cc:2410] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:09.466897 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.484093 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.502373 26337 raft_consensus.cc:2376] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6. Re-sending same reply.
I20260709 12:19:09.380744 29503 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:09.405167 29393 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:09.446390 28983 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.478627 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.472843 29578 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:09.506412 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.506101 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:09.506065 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 629) took 1002 ms. Trace:
I20260709 12:19:09.512208 25629 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:09.515584 25633 raft_consensus.cc:2368] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:09.519061 25785 rpcz_store.cc:276] 0709 12:19:08.504056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.508086 (+  4030us) service_pool.cc:224] Handling call
0709 12:19:09.225927 (+717841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.429007 (+203080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.505903 (+ 76896us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54001a77d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:09.506014 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534686976}
I20260709 12:19:09.500168 25780 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:09.522310 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:09.523049 25704 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:09.512857 29298 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.544942 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.520495 25783 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:09.488242 29495 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.547120 25780 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:09.503408 29550 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:09.438585 29511 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.559579 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.472051 29321 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.513832 29503 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:09.563695 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 491203us
I20260709 12:19:09.512826 29455 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.572768 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.599472 26339 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:09.524995 29593 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.603432 25711 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 12:19:09.603793 25711 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:09.572407 25710 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:09.607426 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.494120 29594 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.561893 29495 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:09.611229 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.619616 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:09.518175 29375 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:09.620815 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 525246us
I20260709 12:19:09.623299 29375 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.627106 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:09.602408 29503 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:09.627853 26335 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:09.521010 28983 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.637123 25636 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:19:09.638275 25637 pending_rounds.cc:85] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 7
I20260709 12:19:09.639122 25637 pending_rounds.cc:107] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:19:09.639452 25637 raft_consensus.cc:2889] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:09.640182 29594 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.640501 29571 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:09.641039 25782 raft_consensus.cc:2368] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:09.620600 29090 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:09.626770 29593 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.663720 25780 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:19:09.620625 26339 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:09.638510 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.527374 29245 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.661788 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.662410 25705 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:09.662950 26337 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:09.666811 25636 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:09.656311 25709 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:09.638406 29511 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:09.676769 25633 raft_consensus.cc:2410] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:19:09.684257 28983 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:09.670925 29509 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:09.685230 29509 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:09.686789 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:09.686144 29509 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.689525 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:09.692821 25710 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:09.694517 25635 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:09.543695 29298 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:09.701557 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.701120 29562 raft_consensus.cc:2764] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:19:09.702479 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 345531us
I20260709 12:19:09.702839 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.703017 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.706030 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.706640 25633 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:09.712564 25636 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:09.725581 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:09.726233 26335 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:09.729574 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:09.734124 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 217233us
I20260709 12:19:09.735694 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:09.747910 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.707494 29513 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.756143 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.757087 25633 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:09.757711 29298 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:09.766764 29022 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.951s	user 0.012s	sys 0.001s
I20260709 12:19:09.767897 25778 raft_consensus.cc:2410] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:09.768501 29459 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.768939 29459 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.729058 29550 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:09.770830 26335 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:09.775446 25710 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
W20260709 12:19:09.814127 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.817052 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 92027.8us
I20260709 12:19:09.706550 29642 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 12:19:09.858762 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.779163 29613 raft_consensus.cc:2764] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:19:09.861152 25677 leader_election.cc:400] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:09.870263 25677 leader_election.cc:403] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 12:19:09.833338 29645 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:19:09.818527 29455 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:09.898797 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:09.908370 25630 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.712s	user 0.012s	sys 0.001s
I20260709 12:19:09.777282 29513 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:09.911672 25630 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:09.916555 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:09.868954 29321 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:09.924851 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 171345us
W20260709 12:19:09.926385 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 220040us
I20260709 12:19:09.936887 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:09.937961 25710 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:09.945623 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 65539.9us
I20260709 12:19:09.893262 29578 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:09.905339 28983 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:09.959079 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:09.931804 29594 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:09.937386 29642 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:09.935087 29090 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:09.940925 29487 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.982463 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.989339 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.001932 29646 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:19:10.003131 29646 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:10.003592 29646 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:10.004624 29646 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.006651 29655 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:19:10.008085 29650 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:19:10.022575 29613 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:10.023458 29578 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.037233 29487 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.041667 29444 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:10.025688 28983 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.047488 29669 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.048012 29669 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.049346 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 100720us
I20260709 12:19:10.049508 29441 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:10.050279 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:10.052021 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.053220 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.050102 29669 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:10.030269 29245 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.130478 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 638) took 1269 ms. Trace:
I20260709 12:19:10.138162 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:10.141471 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 83108.9us
I20260709 12:19:10.142045 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 643) took 1029 ms. Trace:
W20260709 12:19:10.143522 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 147932us
I20260709 12:19:10.023742 29613 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:09.973929 29022 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:10.131695 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:10.175595 25711 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:10.056872 29655 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:10.134582 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 640) took 1190 ms. Trace:
I20260709 12:19:10.142812 25784 rpcz_store.cc:276] 0709 12:19:09.113198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.619687 (+506489us) service_pool.cc:224] Handling call
0709 12:19:09.620016 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.620224 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:09.940032 (+319808us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.141618 (+201586us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.141682 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:10.141745 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.141832 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.141978 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2352,"prepare.queue_time_us":422521,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":49,"thread_start_us":516816,"threads_started":2}
I20260709 12:19:10.137648 25779 rpcz_store.cc:276] 0709 12:19:08.861124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.861230 (+   106us) service_pool.cc:224] Handling call
0709 12:19:08.879909 (+ 18679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.880091 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.880108 (+    17us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.934040 (+ 53932us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.424632 (+490592us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:09.425532 (+   900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.425622 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.054820 (+629198us) raft_consensus.cc:1685] finished
0709 12:19:10.054842 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.130381 (+ 75539us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":321783,"threads_started":1,"wal-append.queue_time_us":626887}
W20260709 12:19:10.183188 25632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1865) took 2033 ms (client timeout 1652 ms). Trace:
I20260709 12:19:10.146376 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:10.183909 25632 rpcz_store.cc:269] 0709 12:19:08.105138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.256969 (+151831us) service_pool.cc:224] Handling call
0709 12:19:10.137832 (+1880863us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54002ac1d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:10.138003 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2084315904}
I20260709 12:19:10.185897 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:10.185865 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:10.129549 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:10.192629 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48847.4us
I20260709 12:19:10.194809 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:10.196188 25706 raft_consensus.cc:2410] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:19:10.196547 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:10.200142 25630 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
W20260709 12:19:10.205408 26339 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.002s
I20260709 12:19:10.205974 26339 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:10.184085 25786 rpcz_store.cc:276] 0709 12:19:08.944140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.553957 (+609817us) service_pool.cc:224] Handling call
0709 12:19:09.579435 (+ 25478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.619151 (+ 39716us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:19:09.619182 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.619705 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.910874 (+291169us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:09.912857 (+  1983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.912976 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.134337 (+221361us) raft_consensus.cc:1685] finished
0709 12:19:10.134358 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.134506 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2301,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":290137,"threads_started":1,"wal-append.queue_time_us":416686}
I20260709 12:19:10.198266 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.197638 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.214612 25709 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:19:10.216682 26336 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:10.210377 26343 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:10.194809 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.222955 25631 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:10.226547 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:10.227944 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:09.981884 29594 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.139565 29508 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:19:10.238639 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 109232us
W20260709 12:19:10.193586 29331 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.801s	user 0.009s	sys 0.009s
I20260709 12:19:10.050294 29441 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:10.266573 26337 raft_consensus.cc:2410] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:19:10.249364 25711 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:10.243074 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:10.267231 29441 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.149335 29375 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:10.267649 29315 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:19:10.268412 29441 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:10.215183 25709 pending_rounds.cc:85] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 6
I20260709 12:19:10.269029 25709 pending_rounds.cc:107] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:19:10.269287 25709 raft_consensus.cc:2889] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:10.262349 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.270045 29315 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:10.264895 29663 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:10.265496 25603 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:10.268349 29375 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:10.264101 29414 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:10.271294 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:10.252915 29508 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:10.303188 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 117242us
I20260709 12:19:10.157979 29579 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:10.176231 29613 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:10.258261 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:10.284178 25603 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:10.284955 29664 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:19:10.286520 29503 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.003s
W20260709 12:19:10.309926 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 255442us
I20260709 12:19:10.270574 29444 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.274809 29646 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:10.325136 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:10.298063 29375 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.299748 29429 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:10.277001 29315 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.328722 25751 leader_election.cc:400] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:10.311632 29550 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.344955 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:10.318280 29545 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.000s	sys 0.015s
I20260709 12:19:10.277103 29331 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:10.366762 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45398.7us
I20260709 12:19:10.321053 29414 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:10.357657 29315 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:10.309563 29022 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.371146 29545 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.000s	sys 0.016s
I20260709 12:19:10.276517 29484 raft_consensus.cc:2955] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:10.367048 29568 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:10.325618 29653 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:19:10.385264 29613 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.377873 29664 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:10.386848 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 641) took 1438 ms. Trace:
I20260709 12:19:10.388489 29593 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.388923 29656 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:10.389122 29594 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:10.369925 29429 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:10.416389 25783 rpcz_store.cc:276] 0709 12:19:08.947985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.546565 (+598580us) service_pool.cc:224] Handling call
0709 12:19:09.547141 (+   576us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:09.547678 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.547703 (+    25us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:09.643263 (+ 95560us) log.cc:844] Serialized 78 byte log entry
0709 12:19:09.935131 (+291868us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:09.935180 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.935262 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.386505 (+451243us) raft_consensus.cc:1685] finished
0709 12:19:10.386541 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.386780 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5963,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":8376,"thread_start_us":252876,"threads_started":1,"wal-append.queue_time_us":704336}
I20260709 12:19:10.394702 25704 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:10.381572 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:10.399508 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 173058us
I20260709 12:19:10.406245 29245 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.364506 29513 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:10.353479 29444 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.429252 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:10.372502 29550 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.411475 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 421132us
I20260709 12:19:10.325896 29503 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.423553 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:10.434991 25783 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:10.437029 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:10.437394 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:10.384286 29651 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:10.425426 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 108033us
I20260709 12:19:10.400914 29545 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:10.414361 29352 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:10.438438 25783 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:10.450230 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.418957 29568 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:10.428509 29321 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:10.477663 25630 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:10.479350 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:10.435856 29593 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:19:10.438818 29495 consensus_meta.cc:287] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.808s	user 0.014s	sys 0.003s
I20260709 12:19:10.451308 29652 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:10.480095 25629 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:10.455806 29613 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:10.490067 29331 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:10.452836 29513 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:19:10.433228 29641 consensus_queue.cc:1175] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "955233c139344131a847044f17f8d348" responder_term: 8 status { last_received { term: 3 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: [4.6-7.7]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:19:10.496912 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 115442us
I20260709 12:19:10.378450 29508 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.513455 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 63318.3us
W20260709 12:19:10.503685 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 178616us
I20260709 12:19:10.433933 29245 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.474915 29352 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:10.511735 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:10.484232 29593 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:10.514236 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.524289 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:10.496606 28983 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.526036 25709 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:10.496240 29682 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:10.503098 29670 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:10.514721 29683 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.527820 29352 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.528964 29503 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.529525 29593 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.531756 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:10.534647 29678 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:19:10.534842 29568 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.525504 29652 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:10.474255 29414 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.525309 29513 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:10.451396 29651 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.543715 29614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.022s	sys 0.005s
I20260709 12:19:10.528129 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:10.548641 25783 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:10.556499 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.493781 29495 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:10.568183 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.543144 29654 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:10.559208 29508 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:10.546700 29653 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:10.548524 29331 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:10.603410 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:10.604105 25783 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:10.608074 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 80026.5us
I20260709 12:19:10.609344 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:10.612195 29654 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.554558 29545 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.614693 29022 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.558894 29683 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.624150 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:10.598201 29614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.023s	sys 0.005s
I20260709 12:19:10.638319 25787 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:10.516567 29664 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.578846 28983 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:19:10.642668 29022 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:19:10.650220 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1832) took 1715 ms (client timeout 1610 ms). Trace:
W20260709 12:19:10.656745 25781 rpcz_store.cc:269] 0709 12:19:08.923028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.036654 (+113626us) service_pool.cc:224] Handling call
0709 12:19:10.637832 (+1601178us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400320854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:10.637969 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842690816}
I20260709 12:19:10.656231 25783 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:10.658416 25781 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:19:10.662182 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:10.662942 25629 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:10.663074 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.551892 29429 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.667837 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 111434us
I20260709 12:19:10.669656 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.651399 29331 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:10.671888 29654 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.672331 29683 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:10.672969 29513 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.674345 29550 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.674685 29686 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:10.675284 29686 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:10.656303 29671 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.329s	user 0.002s	sys 0.000s
I20260709 12:19:10.679940 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.677945 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:10.686142 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:10.698446 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.666548 29699 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.680145 29550 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:19:10.735133 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 70053.5us
I20260709 12:19:10.735488 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:10.741834 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51730.4us
W20260709 12:19:10.704237 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 729393us
I20260709 12:19:10.681347 29444 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:10.723052 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:10.704329 25779 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:10.781045 25629 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:10.696519 29510 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:10.744048 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.692684 29663 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:19:10.797706 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:10.798676 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63315.7us
I20260709 12:19:10.799331 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:10.801762 29348 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:10.805881 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:10.743439 29602 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:10.780675 29683 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:10.791074 29643 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:10.741336 29090 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:10.819473 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2208) took 1187 ms. Trace:
I20260709 12:19:10.824612 25637 rpcz_store.cc:276] 0709 12:19:09.631974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.637538 (+  5564us) service_pool.cc:224] Handling call
0709 12:19:09.638028 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.640242 (+  2214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.640289 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.640890 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.939557 (+298667us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:10.319965 (+380408us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.350776 (+ 30811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.350889 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.655609 (+304720us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:10.812793 (+157184us) raft_consensus.cc:1685] finished
0709 12:19:10.819157 (+  6364us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.819382 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1032,"prepare.queue_time_us":536135,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":41181,"spinlock_wait_cycles":45557504,"thread_start_us":654977,"threads_started":2,"wal-append.queue_time_us":581518}
W20260709 12:19:10.822139 25782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 639) took 1944 ms (client timeout 1605 ms). Trace:
W20260709 12:19:10.827595 25782 rpcz_store.cc:269] 0709 12:19:08.878217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.907135 (+ 28918us) service_pool.cc:224] Handling call
0709 12:19:10.821912 (+1914777us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54002b5554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:10.822058 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1602516352}
W20260709 12:19:10.828895 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 84075.6us
I20260709 12:19:10.792238 29510 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:10.832329 29510 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:10.834477 29613 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:10.727272 29652 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.837942 29669 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:10.859858 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2481) took 1894 ms. Trace:
I20260709 12:19:10.811651 29654 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:19:10.839835 29649 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.891s	user 0.003s	sys 0.000s
I20260709 12:19:10.840471 29595 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:19:10.841408 29646 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.000s
I20260709 12:19:10.842797 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:10.843204 29692 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:10.873980 25779 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:10.874451 25635 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:10.874151 29692 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.844246 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:10.845701 29651 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:10.846374 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:10.846890 29657 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:19:10.846937 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2318) took 2480 ms (client timeout 2998 ms). Trace:
I20260709 12:19:10.847786 29700 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:19:10.850385 29691 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 12:19:10.834754 29701 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:10.807119 29245 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:10.851872 25786 raft_consensus.cc:2955] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:10.838575 29679 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:10.831743 29683 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.879251 25782 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:19:10.879357 29679 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:10.879958 29679 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:10.880689 25782 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:19:10.880934 29683 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:10.724534 29699 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:10.862841 29321 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:10.853299 29655 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.009s
I20260709 12:19:10.854549 29578 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:10.881084 25782 pending_rounds.cc:85] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 6
I20260709 12:19:10.895512 25782 pending_rounds.cc:107] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:19:10.860327 25705 rpcz_store.cc:276] 0709 12:19:08.959483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.969915 (+ 10432us) service_pool.cc:224] Handling call
0709 12:19:10.849035 (+1879120us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54001bf154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:10.853521 (+  4486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440817024}
W20260709 12:19:10.878077 25778 rpcz_store.cc:269] 0709 12:19:08.366198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.384651 (+ 18453us) service_pool.cc:224] Handling call
0709 12:19:10.846726 (+2462075us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000b04d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:10.846882 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3012796288}
I20260709 12:19:10.893113 29699 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:10.895903 25782 raft_consensus.cc:2889] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:10.859926 29669 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:10.903656 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 109477us
I20260709 12:19:10.864914 29654 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:10.906725 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 216681us
I20260709 12:19:10.878172 29657 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:10.920933 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:10.921888 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 287178us
I20260709 12:19:10.870213 29595 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:10.894583 29495 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:10.900679 29655 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:10.942819 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:10.900456 29550 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:10.948042 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.917311 29654 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:10.947348 26307 leader_election.cc:400] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:10.957237 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.957899 25638 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:10.958082 26338 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:10.958745 26307 leader_election.cc:403] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:19:10.960095 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:10.962265 25677 connection.cc:256] RPC call timeout handler was delayed by -1.58685679600012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
I20260709 12:19:10.963385 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:10.963912 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.965031 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:10.967746 26336 raft_consensus.cc:2368] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:10.970571 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.977886 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:10.954144 29683 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:10.961886 29550 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:10.982156 29550 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:10.983398 29696 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:11.016006 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:10.979485 25677 leader_election.cc:400] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10.
I20260709 12:19:10.983633 29487 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:11.032270 25638 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:19:11.018944 29709 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:19:11.018733 29595 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:11.020301 29375 consensus_meta.cc:287] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.000s
I20260709 12:19:11.022830 29686 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:11.090294 26336 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:10.963064 29090 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.030292 25677 leader_election.cc:403] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:19:11.034821 25782 raft_consensus.cc:2368] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10.
W20260709 12:19:11.035380 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 235731us
I20260709 12:19:11.026340 29331 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:10.972213 29663 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:11.114699 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:11.114916 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.085268 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:11.121791 29707 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
W20260709 12:19:11.093252 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.112303 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.122881 29646 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10.
I20260709 12:19:11.130777 29655 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:11.149201 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.149513 29655 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.150005 29656 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.112536 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:11.118706 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 139446us
I20260709 12:19:11.089773 29682 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.151425 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.151539 29682 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.152607 29414 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:11.152671 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2430) took 1018 ms. Trace:
I20260709 12:19:11.153211 29414 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:11.153842 29414 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:11.153908 29682 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:11.083472 29701 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:11.154821 29701 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.157097 29701 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:11.125178 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.158743 29655 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:11.024323 29315 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.000s
I20260709 12:19:11.160022 29315 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:11.161388 29717 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:19:11.163655 29441 consensus_meta.cc:287] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.895s	user 0.006s	sys 0.007s
W20260709 12:19:11.164541 29508 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.005s
I20260709 12:19:11.153275 25704 rpcz_store.cc:276] 0709 12:19:10.133690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.209921 (+ 76231us) service_pool.cc:224] Handling call
0709 12:19:10.228745 (+ 18824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.426097 (+197352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.152338 (+726241us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002ec2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:11.152507 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381178752}
I20260709 12:19:11.090291 29487 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:11.166088 29487 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.128502 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.118584 29711 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.092330 29642 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:11.134459 25786 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:11.114229 29090 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.117012 29444 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:11.125388 29298 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.145219 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:11.145673 29708 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.131s	user 0.012s	sys 0.004s
I20260709 12:19:11.146049 29595 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:11.146184 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:11.079945 29657 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.091492 29375 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.162329 29315 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:11.172094 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.173935 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:11.173370 29705 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.270s	user 0.002s	sys 0.000s
I20260709 12:19:11.184737 25638 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:11.187471 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:11.210342 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 44119.2us
I20260709 12:19:11.180984 29444 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.202108 26338 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:11.122342 29706 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.246s
I20260709 12:19:11.237262 25784 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:11.229573 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:11.243510 25786 raft_consensus.cc:2376] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8. Re-sending same reply.
I20260709 12:19:11.238431 25705 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:19:11.239159 25710 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 12:19:11.245471 25708 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:19:11.246201 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2323) took 2295 ms. Trace:
W20260709 12:19:11.249241 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.249119 25780 rpcz_store.cc:276] 0709 12:19:08.950815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.561147 (+610332us) service_pool.cc:224] Handling call
0709 12:19:09.561455 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.664165 (+102710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.244777 (+1580612us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b5400099a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:11.246141 (+  1364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191735680}
I20260709 12:19:11.252110 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.254011 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.182435 29298 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.174770 29642 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.257400 26336 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
I20260709 12:19:11.254024 25780 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:11.258867 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.260190 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.261039 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:11.263268 29716 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.147s	user 0.007s	sys 0.003s
I20260709 12:19:11.266677 25711 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:11.268167 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:11.268627 29298 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:11.268805 25711 raft_consensus.cc:2368] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:11.269531 25710 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:11.269739 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.270604 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:11.271095 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.271229 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:11.271441 25712 raft_consensus.cc:2410] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:19:11.271895 25711 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:19:11.272327 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.273213 25634 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:11.275676 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.277523 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:11.278743 29720 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:11.278827 29722 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:11.279726 29722 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:11.279803 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:11.281040 25630 raft_consensus.cc:2410] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:19:11.281126 25713 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:11.281543 29375 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:11.280165 29722 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:11.282819 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.283663 26340 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:11.284112 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.284749 25637 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:11.285604 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:11.289614 29722 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.291291 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.294706 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.282305 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.296474 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:11.297389 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.274269 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:11.298431 25634 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:11.298480 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:11.173604 29711 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.304607 25638 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:19:11.306396 25708 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:11.297762 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.312165 25710 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:19:11.307539 25711 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:11.308314 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:11.313719 25780 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:11.229923 29701 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:11.311151 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:11.291327 29723 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:19:11.315723 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:11.326910 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49550.2us
I20260709 12:19:11.327553 25637 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:19:11.325802 25786 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:19:11.326979 25705 raft_consensus.cc:2376] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7. Re-sending same reply.
I20260709 12:19:11.330286 25707 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:11.341078 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1841) took 2415 ms (client timeout 2471 ms). Trace:
I20260709 12:19:11.343739 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:11.341686 25787 rpcz_store.cc:269] 0709 12:19:08.925651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.390865 (+465214us) service_pool.cc:224] Handling call
0709 12:19:11.340829 (+1949964us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400307fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:11.340995 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2755279744}
W20260709 12:19:11.346835 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.349232 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:11.355583 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.357564 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:11.372213 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:11.379190 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:11.380079 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:11.388561 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 44946.1us
W20260709 12:19:11.381407 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.388063 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:11.392351 25637 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:11.396075 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.369762 29720 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:11.369341 29664 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:11.377872 29682 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:11.387997 29679 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.002s
I20260709 12:19:11.354801 29701 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.403940 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.391847 29698 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:11.413688 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.417307 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.418219 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.373672 29594 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:19:11.424981 25633 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:19:11.425223 25638 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:19:11.406206 29487 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:11.409746 29682 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.500837 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:11.498970 25708 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:19:11.503545 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.504662 25637 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:11.423589 29698 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.517845 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.518958 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.519093 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.519641 25705 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:11.497768 29664 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:19:11.406653 29720 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.413738 29679 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.519641 25634 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:11.512058 29682 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:11.540963 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.516422 29698 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:11.559615 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 170201us
I20260709 12:19:11.564782 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:11.569806 25630 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:11.575884 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.561728 29594 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:11.594887 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.596662 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:11.603017 25631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2196) took 2401 ms (client timeout 2006 ms). Trace:
W20260709 12:19:11.603571 25631 rpcz_store.cc:269] 0709 12:19:09.195920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.229401 (+ 33481us) service_pool.cc:224] Handling call
0709 12:19:11.597504 (+2368103us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400364a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:11.597641 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582628736}
W20260709 12:19:11.608366 29510 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.775s	user 0.010s	sys 0.000s
I20260709 12:19:11.608845 29510 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.618868 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:11.619784 25705 raft_consensus.cc:2368] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:11.571324 29664 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:11.627979 25780 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:11.625018 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.633384 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 132263us
W20260709 12:19:11.642099 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45546.3us
I20260709 12:19:11.643967 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:11.645551 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:11.596171 29656 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:11.623528 29726 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:19:11.661554 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.593438 29090 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:11.624702 29725 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 12:19:11.623142 29487 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.632408 29657 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:11.676590 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.689113 25710 raft_consensus.cc:2376] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7. Re-sending same reply.
I20260709 12:19:11.605329 29642 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:11.651814 29654 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
I20260709 12:19:11.674252 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.682256 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:11.660797 29728 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 12:19:11.623445 29641 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.639528 29711 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:11.697034 25634 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:11.645861 29315 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:19:11.712386 25706 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:11.705843 29654 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.633545 29594 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.698518 29722 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:11.702840 29656 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.644202 25780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:11.729979 25780 raft_consensus.cc:606] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.730509 25780 raft_consensus.cc:614] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:11.732333 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:11.734046 25705 raft_consensus.cc:2376] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8. Re-sending same reply.
W20260709 12:19:11.717666 29689 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.184s	user 0.030s	sys 0.025s
I20260709 12:19:11.622145 29664 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.745724 25779 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:11.750027 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 106161us
I20260709 12:19:11.772861 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:11.694093 29487 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.776079 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:11.757934 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:11.755990 29730 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:19:11.767393 29689 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.030s	sys 0.026s
I20260709 12:19:11.755981 29654 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:11.792577 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:11.785542 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:11.788753 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:11.786099 25751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.969s (SENT) [suppressed 40 similar messages]
I20260709 12:19:11.794348 25706 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:11.793510 25705 raft_consensus.cc:2376] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7. Re-sending same reply.
W20260709 12:19:11.792105 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 146681us
I20260709 12:19:11.752476 29701 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:11.747947 29732 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:11.798049 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:11.803215 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 94130.1us
I20260709 12:19:11.787770 29656 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:11.803508 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:11.806136 29487 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:11.800690 29414 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.005s
I20260709 12:19:11.810536 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:11.813135 29721 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.666s	user 0.001s	sys 0.000s
I20260709 12:19:11.843586 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:11.808559 29687 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.266s	user 0.003s	sys 0.000s
I20260709 12:19:11.792354 29722 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:19:11.846428 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.820183 29393 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:19:11.820998 29663 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.001s
I20260709 12:19:11.802924 29510 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:11.803918 29720 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:11.802502 29743 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:11.822831 25705 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:19:11.816902 29732 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.825706 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.853889 25634 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:11.854550 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 833) took 2140 ms. Trace:
I20260709 12:19:11.826395 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.836203 29660 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:11.856523 29732 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:11.855163 25636 rpcz_store.cc:276] 0709 12:19:09.714460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.753909 (+ 39449us) service_pool.cc:224] Handling call
0709 12:19:09.754607 (+   698us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:09.764700 (+ 10093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.764742 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:09.765663 (+   921us) log.cc:844] Serialized 78 byte log entry
0709 12:19:10.225803 (+460140us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:10.405645 (+179842us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.416068 (+ 10423us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.324778 (+908710us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b180039aa20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:11.558976 (+234198us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.849371 (+290395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.854280 (+  4909us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.854326 (+    46us) raft_consensus.cc:1685] finished
0709 12:19:11.854339 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.854469 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36326,"prepare.queue_time_us":633862,"prepare.run_cpu_time_us":176,"prepare.run_wall_time_us":21611,"raft.queue_time_us":66232,"raft.run_cpu_time_us":581,"raft.run_wall_time_us":38749,"spinlock_wait_cycles":2721024,"thread_start_us":1055715,"threads_started":6,"wal-append.queue_time_us":743540,"wal-append.run_cpu_time_us":5411,"wal-append.run_wall_time_us":1242690}
I20260709 12:19:11.856910 29660 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.857753 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:11.858053 29739 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:19:11.858948 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.830092 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.859371 29663 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:11.859870 25630 raft_consensus.cc:2368] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:19:11.849208 29393 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:11.860704 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.861720 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:11.860911 29393 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.862174 25630 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:11.852088 29743 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.862818 29743 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:11.836827 25778 raft_consensus.cc:1217] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Deduplicated request from leader. Original: 7.8->[9.9-9.9]   Dedup: 9.9->[]
I20260709 12:19:11.864773 25778 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:19:11.835451 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.848831 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:11.865639 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61079.5us
I20260709 12:19:11.866250 29510 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.866386 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.867215 26338 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:19:11.850461 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:11.868685 26340 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:11.868894 26341 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:11.859138 29660 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:11.852268 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:11.855183 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.829006 29594 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:11.872200 25630 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:11.873473 25634 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:11.873905 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.874259 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.874552 25630 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:11.875353 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.876348 25630 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:11.880502 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:11.881975 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.879244 29393 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:11.849596 29441 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:11.888304 25637 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:11.892071 25636 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:11.894765 29441 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.901908 29022 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:11.902398 29022 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:11.902840 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.868381 29619 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:11.902880 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:11.837459 29414 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.916394 25633 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:11.906383 29619 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:11.918406 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.920536 25601 connection.cc:256] RPC call timeout handler was delayed by -1.396568464999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2998s for a response
I20260709 12:19:11.920949 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:11.921752 25783 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:11.922047 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:11.925948 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.927199 29745 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.933422 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:11.934154 25783 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:11.936074 29619 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.937413 29619 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:11.937973 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.941547 29444 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:11.949031 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.932312 29664 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:11.957472 25751 leader_election.cc:400] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:19:11.946719 29660 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:19:11.963271 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:11.911391 29022 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.964428 25783 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:11.948042 29352 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:11.969548 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:11.971007 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.974666 29653 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:11.974820 25781 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:19:11.975030 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:11.975304 29653 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:11.976058 29746 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.976550 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2343) took 2051 ms. Trace:
I20260709 12:19:11.976184 29653 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:11.976962 25781 rpcz_store.cc:276] 0709 12:19:09.926133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.659137 (+733004us) service_pool.cc:224] Handling call
0709 12:19:10.659495 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:11.975350 (+1315855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.976244 (+   894us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400098654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:11.976486 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3024106240}
I20260709 12:19:11.977806 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1844) took 1067 ms. Trace:
I20260709 12:19:11.978364 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.978716 29487 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:11.979753 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:11.980005 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:11.971280 29745 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.981462 29487 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:11.981917 29487 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.982326 29487 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:11.983462 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1847) took 1022 ms. Trace:
I20260709 12:19:11.983841 25787 rpcz_store.cc:276] 0709 12:19:10.960752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.568775 (+608023us) service_pool.cc:224] Handling call
0709 12:19:11.594550 (+ 25775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.844095 (+249545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:11.844130 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:11.844177 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:11.850178 (+  6001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.983212 (+133034us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54001a63d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:11.983261 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.983407 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551734016}
I20260709 12:19:11.985394 25787 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:19:11.982018 29719 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.815s	user 0.000s	sys 0.003s
I20260709 12:19:11.978186 25778 rpcz_store.cc:276] 0709 12:19:10.910398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.910534 (+   136us) service_pool.cc:224] Handling call
0709 12:19:10.910932 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:11.865116 (+954184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.977181 (+112065us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b5400249120. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5906dc0 00007f29b5992a02 00007f29b598edd6 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:19:11.977596 (+   415us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400086bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:11.977749 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144560256}
I20260709 12:19:11.989300 25783 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:19:11.991734 25778 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:19:11.991925 29743 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:11.990576 29022 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:11.995050 25634 raft_consensus.cc:2368] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:19:11.996490 25778 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:11.998319 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:11.965529 29660 consensus_queue.cc:1243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:19:12.002928 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.004071 29683 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.004777 29683 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.005694 25638 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:12.004814 25631 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:19:12.007570 29683 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:12.008848 29669 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.013567 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 650) took 1227 ms. Trace:
I20260709 12:19:11.971998 29664 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:12.019654 25779 rpcz_store.cc:276] 0709 12:19:10.786611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.875726 (+ 89115us) service_pool.cc:224] Handling call
0709 12:19:10.875970 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.772017 (+896047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.013273 (+241256us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54000866d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:12.013494 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1999662976}
I20260709 12:19:11.970064 29352 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:12.021991 29663 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:19:12.026561 29663 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:12.027037 29663 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:12.028234 29663 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.029691 29663 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:12.021283 29743 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.034842 29690 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:12.035310 29690 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:12.036312 29690 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:12.039156 29714 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.893s	user 0.003s	sys 0.000s
I20260709 12:19:12.015741 29682 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.042037 29593 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:12.045780 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.022843 25603 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.051414 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:12.046609 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:12.023128 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:12.052299 25779 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:12.039814 29664 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.048522 29669 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:12.054049 29683 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:12.054652 29683 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.054752 29743 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:12.055106 29683 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:12.055542 29745 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:12.056197 25786 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:12.056808 29682 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.017637 29698 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.058791 29698 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.061009 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:12.061919 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:12.062690 26335 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:12.055279 29743 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.064219 29743 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:12.066803 29706 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:12.066891 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:12.067447 29706 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.067638 26343 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.067914 29706 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:12.071156 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:12.080710 25781 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:12.081830 29745 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:12.084157 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.076900 26335 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
W20260709 12:19:12.089874 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67157.2us
I20260709 12:19:12.089360 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:12.087174 29690 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:12.107318 25778 pending_rounds.cc:85] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 4
I20260709 12:19:12.096662 29657 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:19:12.110546 29692 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.021838 29352 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.114142 25778 pending_rounds.cc:107] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:19:12.120733 25778 raft_consensus.cc:2889] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:12.122605 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.122402 25779 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:12.130089 29680 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:12.113564 29508 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:19:12.139366 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.136271 29680 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:12.128434 29455 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:19:12.123659 29513 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:19:12.156311 26343 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:12.157089 25637 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:19:12.158174 25637 pending_rounds.cc:85] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 5
I20260709 12:19:12.158444 25637 pending_rounds.cc:107] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:19:12.158514 29697 raft_consensus.cc:2764] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:19:12.159176 25637 raft_consensus.cc:2889] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:12.128647 29444 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 12:19:12.164911 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 367136us
I20260709 12:19:12.165699 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:12.165769 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 362292us
I20260709 12:19:12.166477 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:12.167109 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:12.167452 29642 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:12.168032 29642 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.168478 29642 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:12.168947 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:12.170823 29744 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:19:12.171021 25784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2340) took 2267 ms (client timeout 1584 ms). Trace:
W20260709 12:19:12.171553 25784 rpcz_store.cc:269] 0709 12:19:09.903138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.318250 (+415112us) service_pool.cc:224] Handling call
0709 12:19:12.170805 (+1852555us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b54000ee054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:12.170952 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2008833792}
I20260709 12:19:12.173931 29331 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:12.174988 29740 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:12.175262 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.175973 29737 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.150328 29090 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:12.175633 29740 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.176642 29737 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.177503 25712 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:19:12.176658 29090 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.178802 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2336) took 2277 ms. Trace:
W20260709 12:19:12.164911 29750 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:19:12.179934 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.180817 29742 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.181373 29742 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.182943 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.184156 29742 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.167326 29641 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:12.185174 29641 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:12.185549 29641 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:12.165091 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 965) took 1030 ms. Trace:
I20260709 12:19:12.176329 25780 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:12.179296 25708 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:19:12.190888 25710 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:12.187659 25677 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:12.184201 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.192662 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:12.144560 29429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.015s	sys 0.012s
I20260709 12:19:12.186353 26336 rpcz_store.cc:276] 0709 12:19:11.134977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.135473 (+   496us) service_pool.cc:224] Handling call
0709 12:19:12.164931 (+1029458us) spinlock_profiling.cc:243] Waited 79.7 ms on lock 0x7b5400498254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:12.165036 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183331200}
I20260709 12:19:12.254938 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.193653 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:12.193956 25676 leader_election.cc:400] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 6. Current term is 7.
I20260709 12:19:12.179263 25785 rpcz_store.cc:276] 0709 12:19:09.901926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.909222 (+  7296us) service_pool.cc:224] Handling call
0709 12:19:09.936317 (+ 27095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.287844 (+351527us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:10.319520 (+ 31676us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.503945 (+184425us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.503999 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:10.504045 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.849677 (+345632us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:12.178544 (+1328867us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54002576d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:12.178595 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.178722 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17095,"prepare.queue_time_us":36653,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":3284,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":6157,"spinlock_wait_cycles":814587648,"thread_start_us":142730,"threads_started":1,"wal-append.queue_time_us":211092,"wal-append.run_cpu_time_us":2955,"wal-append.run_wall_time_us":1349967}
I20260709 12:19:12.165145 29701 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:12.195159 25710 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:12.155625 29352 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:12.243014 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.243906 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.243024 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.260123 25784 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:12.260375 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.179100 29090 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:12.243050 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48795us
I20260709 12:19:12.195297 25708 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:19:12.247771 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.182128 29487 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.257313 25711 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:19:12.256948 25704 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
W20260709 12:19:12.269470 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 77900.7us
I20260709 12:19:12.270648 25708 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:19:12.261055 25710 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:12.263484 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.266825 25632 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:12.271299 25708 pending_rounds.cc:85] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 4
I20260709 12:19:12.233206 29740 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.271768 25708 pending_rounds.cc:107] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:19:12.272163 25708 raft_consensus.cc:2889] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:12.243220 29331 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:19:12.279160 29429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.017s	sys 0.016s
I20260709 12:19:12.280213 26342 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:12.285370 25712 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:19:12.285956 25712 pending_rounds.cc:85] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 3
I20260709 12:19:12.286224 25712 pending_rounds.cc:107] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:19:12.286651 25712 raft_consensus.cc:2889] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:12.289805 25710 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:12.291836 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.261058 29737 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.300591 25603 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.305190 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.305862 25634 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
W20260709 12:19:12.311511 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48118.4us
I20260709 12:19:12.313336 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:12.314564 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:12.320885 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.322116 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.322851 25634 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:12.323681 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:12.324148 25634 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.307514 29562 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:19:12.306023 29393 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:12.304203 29737 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:12.309358 29567 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:19:12.332170 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:12.322944 29487 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:12.338750 25677 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.341560 29375 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.341949 29711 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:12.342661 29701 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:12.343616 29701 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:12.343962 29701 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:12.345256 29701 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.333115 29619 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:12.318634 29331 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:12.350184 25634 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:12.354049 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1854) took 1321 ms. Trace:
I20260709 12:19:12.355557 29090 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.311074 29732 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:19:12.330695 29656 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.311204 29720 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:19:12.342123 29375 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.361696 29656 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.340313 29740 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:12.361289 25780 rpcz_store.cc:276] 0709 12:19:11.033336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.789209 (+755873us) service_pool.cc:224] Handling call
0709 12:19:12.176090 (+386881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.190819 (+ 14729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.353779 (+162960us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b54000ee050. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:12.353966 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888824704}
I20260709 12:19:12.389446 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:12.391466 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:12.348325 29701 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:12.399844 29656 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.400310 29711 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:12.400609 29090 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.401705 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 653) took 1431 ms. Trace:
W20260709 12:19:12.409530 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.409879 25786 rpcz_store.cc:276] 0709 12:19:10.965706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.378997 (+413291us) service_pool.cc:224] Handling call
0709 12:19:12.396647 (+1017650us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54001a72d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:12.396804 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515824000}
I20260709 12:19:12.401412 29619 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:12.337145 29743 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.411902 29459 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:12.388394 29683 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:12.403656 29751 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:19:12.407893 29331 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.443460 29331 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.443850 29642 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.445252 29641 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.430320 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.402036 29745 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:12.403882 29737 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:19:12.436228 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 78128.5us
I20260709 12:19:12.438558 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.438114 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.452607 25706 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:12.439035 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.434875 26344 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:12.395424 29740 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:19:12.450287 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:12.450368 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.453545 25676 leader_election.cc:400] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10.
I20260709 12:19:12.455600 25713 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:12.445829 29331 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.461994 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.464073 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.469136 25711 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:12.473027 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:12.473852 25633 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:12.438979 29619 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:12.451459 29737 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:12.454465 29513 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:12.446090 29642 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.460549 29090 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.490962 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.474941 29455 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:12.468022 29663 raft_consensus.cc:2764] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:19:12.498286 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.458132 29484 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:19:12.415462 29680 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:12.503886 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.504395 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2212) took 2871 ms. Trace:
I20260709 12:19:12.505498 25635 rpcz_store.cc:276] 0709 12:19:09.633443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.716004 (+ 82561us) service_pool.cc:224] Handling call
0709 12:19:09.734086 (+ 18082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.874875 (+1140789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.504171 (+1629296us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002abcd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:12.504338 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622191360}
I20260709 12:19:12.480268 29692 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:12.506528 26336 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:12.506886 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.507210 26336 raft_consensus.cc:606] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:12.507627 26336 raft_consensus.cc:614] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:12.495955 29737 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:12.509003 29642 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:12.510110 29642 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:12.510432 29642 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:12.511307 29642 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.512432 29642 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:12.508256 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.515479 29331 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:12.517622 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:12.476007 29699 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:19:12.520586 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.521236 25635 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:12.521931 29699 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:12.522845 29513 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:19:12.523159 29758 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:19:12.523940 29441 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.524513 29683 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.476895 29669 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:19:12.526635 29743 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.527339 29683 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.527966 29683 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.528770 29744 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:12.531472 29683 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.534040 25631 raft_consensus.cc:1217] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Deduplicated request from leader. Original: 8.9->[10.10-10.10]   Dedup: 10.10->[]
I20260709 12:19:12.534801 25631 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:19:12.535532 29429 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:19:12.503072 29090 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.485653 29706 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.541733 29641 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.548552 29706 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:12.545327 25713 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:12.546131 25629 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:12.550539 29619 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.565404 29414 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.569954 29744 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:12.570430 29744 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:12.570829 29744 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:12.572330 29613 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:12.572430 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.572774 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.573122 29613 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:12.573983 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.574735 29692 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:12.575131 29692 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:12.576116 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.577737 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.577910 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.579948 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.580854 25707 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:12.554774 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:12.574142 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:12.582968 25709 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:12.575994 29692 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.591603 25638 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:12.591629 25713 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:12.596213 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.597307 25713 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:12.601634 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.554728 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:12.604652 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.584652 25633 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:12.588238 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.607666 25634 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:12.598361 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.524524 29513 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:12.618108 25713 raft_consensus.cc:2410] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:19:12.615687 25705 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:12.615693 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61225.9us
I20260709 12:19:12.620592 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:12.621930 25633 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:12.519935 29455 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
W20260709 12:19:12.615366 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 136576us
W20260709 12:19:12.623775 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.629518 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:12.620008 25709 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:12.630504 25712 raft_consensus.cc:2410] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 12:19:12.615264 29762 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:19:12.634446 25633 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:12.637104 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.614974 29429 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:12.621824 29331 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:12.574110 29613 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.631681 29692 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:12.645558 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:12.647326 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:12.640669 29699 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:12.664561 29509 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:12.667593 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:12.669257 29690 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.001s
I20260709 12:19:12.666754 29646 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:12.671303 29646 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:12.672333 29646 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.673390 29090 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:12.639873 29669 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:19:12.643630 29429 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.678144 29429 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.650942 29331 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:12.679706 29331 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.684791 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.686007 26336 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:12.649925 29740 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:12.686717 26336 raft_consensus.cc:606] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:12.687139 26336 raft_consensus.cc:614] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:12.685163 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:12.690996 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.699472 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 1879) took 1433 ms. Trace:
I20260709 12:19:12.699963 25629 rpcz_store.cc:276] 0709 12:19:11.266084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.303776 (+ 37692us) service_pool.cc:224] Handling call
0709 12:19:12.166326 (+862550us) spinlock_profiling.cc:243] Waited 996 us on lock 0x7b5800000c98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:19:12.699285 (+532959us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54002ab7d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:12.699408 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684130944}
I20260709 12:19:12.701344 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:12.704093 25629 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:19:12.699502 25634 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
W20260709 12:19:12.715925 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 626590us
I20260709 12:19:12.716290 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.716655 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:12.716845 25707 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:12.688544 29740 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.717902 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:12.725996 29090 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:12.726847 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.727306 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.728096 25786 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:19:12.723904 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.729324 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.729656 26309 leader_election.cc:400] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:12.726269 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.776515 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:12.777971 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45979.9us
I20260709 12:19:12.779421 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:12.781261 25780 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:12.783501 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.784866 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.783501 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:12.785998 25784 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:12.789855 25786 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:12.666556 29513 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:12.794878 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 77026.4us
I20260709 12:19:12.795907 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:12.801608 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.718994 29669 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:12.728339 29315 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:12.773926 29699 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:12.732810 29510 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:12.815816 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:12.729106 29766 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:12.724624 29740 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.733958 29655 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:12.719170 29429 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:19:12.812763 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.810981 29742 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:12.825366 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:12.807548 29683 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:12.809978 29352 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:12.811259 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.835215 25785 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:19:12.838444 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:12.822183 29459 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:12.785712 29744 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.812471 29698 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:19:12.812752 29737 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.832827 29683 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.821552 29669 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:19:12.852034 29711 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:12.824600 29510 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.840293 29579 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.875021 29579 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:12.876318 29510 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:12.858911 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 128354us
I20260709 12:19:12.840539 29655 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:12.883416 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.883620 29655 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.845732 29429 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:12.885386 25752 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.886405 29429 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.886920 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.887586 29484 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:12.888019 29484 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:12.888445 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.888847 29484 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:12.868983 29737 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:12.846035 29459 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:12.852496 29321 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:12.872016 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:12.856364 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 71923us
I20260709 12:19:12.895378 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.862906 29683 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:12.898411 25711 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:12.899755 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.901352 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.908129 26339 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:12.908246 26341 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:12.901973 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.903422 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.904852 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:12.909807 26338 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:12.905416 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.910772 26337 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:12.912914 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:12.913879 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.913992 26343 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:19:12.894173 29459 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:12.915695 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.921041 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.939601 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:12.892095 29090 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:12.919031 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:12.966657 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.916787 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:12.943259 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.913838 29706 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:12.919037 29484 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:12.895654 29510 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:19:12.975339 26339 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:19:12.981173 26339 pending_rounds.cc:85] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 7
I20260709 12:19:12.981514 26339 pending_rounds.cc:107] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:19:12.981828 26339 raft_consensus.cc:2889] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:12.968535 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.986238 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.974881 29779 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:13.001941 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:12.995136 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:12.995424 26335 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
W20260709 12:19:12.994889 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 156488us
I20260709 12:19:13.000469 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:13.004474 26340 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:13.007836 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:13.013880 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:13.014132 25785 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:19:13.014511 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:13.032326 25751 leader_election.cc:336] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.006s (SENT)
I20260709 12:19:13.043053 26336 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:13.040886 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.013579 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:19:13.003017 29785 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:13.056022 29646 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:13.056540 26344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 963) took 1940 ms (client timeout 1577 ms). Trace:
I20260709 12:19:13.056620 29646 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:13.057027 26344 rpcz_store.cc:269] 0709 12:19:11.115812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.115957 (+   145us) service_pool.cc:224] Handling call
0709 12:19:13.056348 (+1940391us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54004812d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:13.056447 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2934429952}
I20260709 12:19:13.013312 29779 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:13.058699 29779 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.059345 29779 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.062561 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:13.063234 25631 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:13.065214 29771 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.173s	user 0.002s	sys 0.000s
I20260709 12:19:13.067133 25677 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:13.080765 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.085289 26341 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:19:13.085632 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:13.031801 29679 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.089704 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:13.014667 29619 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:13.111393 29619 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:13.112478 29619 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.113901 29619 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:12.978823 29090 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.083321 29701 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:13.097271 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 64982.2us
I20260709 12:19:13.107822 29789 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:13.124842 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:13.114782 29646 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.132871 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.133205 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:13.133909 25779 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:13.135437 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:13.136142 25779 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.120276 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:13.122153 25785 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:19:13.152359 25785 pending_rounds.cc:85] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 4
I20260709 12:19:13.152796 25785 pending_rounds.cc:107] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:19:13.153401 25785 raft_consensus.cc:2889] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:13.123538 29579 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:13.124084 25633 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:19:13.124179 25780 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:19:13.152637 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66241.1us
I20260709 12:19:13.122901 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:13.162861 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 246192us
I20260709 12:19:13.164083 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:13.164505 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44345.2us
I20260709 12:19:13.165584 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.157611 25782 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:13.169878 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.171733 25780 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:19:13.175402 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:13.126499 29701 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.127908 29352 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:13.185722 29679 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:19:13.199347 29352 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.201601 29513 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:13.205695 29775 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 12:19:13.209141 25779 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
W20260709 12:19:13.209342 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 43841.9us
I20260709 12:19:13.210886 25708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:50836
I20260709 12:19:13.211571 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:13.211941 25708 raft_consensus.cc:606] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Received request to transfer leadership
I20260709 12:19:13.212503 25708 raft_consensus.cc:614] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:13.193163 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:13.214089 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.132047 29711 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:13.216861 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:13.223219 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:13.223951 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.224226 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:13.225077 25704 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:13.227108 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.230039 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:13.232431 25712 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:13.170727 29315 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:13.167683 29414 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.240005 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:13.241293 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 71481.6us
I20260709 12:19:13.241521 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:13.242192 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:13.154747 29782 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:13.193246 29321 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:13.254205 26343 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:19:13.161345 29706 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:19:13.209762 29701 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:13.257551 25713 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:19:13.187651 29744 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.261154 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 658) took 1890 ms. Trace:
W20260709 12:19:13.230240 29692 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.005s
I20260709 12:19:13.167858 29646 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:19:13.218775 29642 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:13.262655 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 956) took 2242 ms. Trace:
I20260709 12:19:13.228513 29742 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.268457 26342 rpcz_store.cc:276] 0709 12:19:11.013190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.013381 (+   191us) service_pool.cc:224] Handling call
0709 12:19:11.016966 (+  3585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.280597 (+1263631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.255276 (+974679us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540036ae54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:13.255437 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2904780160}
I20260709 12:19:13.221283 29711 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:13.270104 25707 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:13.245337 29315 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:13.250630 29782 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.281092 25704 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:13.255726 29487 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.219587 29790 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:19:13.253741 29783 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:13.259246 29698 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:13.262228 25787 rpcz_store.cc:276] 0709 12:19:11.370331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.986174 (+615843us) service_pool.cc:224] Handling call
0709 12:19:11.986431 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.988213 (+  1782us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:12.001088 (+ 12875us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.092790 (+ 91702us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.715658 (+622868us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b18004c0360. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:12.829173 (+113515us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.250067 (+420894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:13.250140 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:13.250203 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.250300 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.261064 (+ 10764us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":678982,"prepare.queue_time_us":212912,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":113,"raft.queue_time_us":676987,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":14326,"spinlock_wait_cycles":4345728,"thread_start_us":614426,"threads_started":3,"wal-append.queue_time_us":144842}
I20260709 12:19:13.265115 29444 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.239413 29683 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.266001 29722 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.296260 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:13.308678 25632 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:13.299775 29790 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:19:13.310276 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.286046 29711 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:13.300303 29375 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:13.288231 29782 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:13.287911 29315 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:13.280871 29510 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.285933 29779 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:13.313131 26338 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:13.288570 29459 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:13.329684 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 174654us
I20260709 12:19:13.295046 29783 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.331095 26341 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:13.332484 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:13.333485 25637 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:13.365681 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.367398 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:13.370299 25784 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:13.373834 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:13.376353 25638 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:13.325717 29792 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:13.312001 29444 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.407935 26340 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:13.318429 29722 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.369484 29090 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:13.457805 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.364790 29783 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:13.537863 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:13.540992 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:13.541780 25710 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:13.552474 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 310366us
I20260709 12:19:13.552816 25636 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:13.326673 29510 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.386080 29790 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:13.561932 25636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2240) took 1374 ms (client timeout 1786 ms). Trace:
I20260709 12:19:13.461052 29759 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:19:13.577071 25636 rpcz_store.cc:269] 0709 12:19:12.187345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.187487 (+   142us) service_pool.cc:224] Handling call
0709 12:19:13.537515 (+1350028us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800000c98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:19:13.561833 (+ 24318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2373888}
W20260709 12:19:13.591019 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 351098us
I20260709 12:19:13.594833 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:13.555045 29773 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 12:19:13.597826 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:13.601528 29777 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.611s	user 0.009s	sys 0.000s
I20260709 12:19:13.609019 29510 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:13.550789 29742 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:13.612682 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.615691 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2356) took 1518 ms. Trace:
I20260709 12:19:13.616186 25778 rpcz_store.cc:276] 0709 12:19:12.097591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.106802 (+  9211us) service_pool.cc:224] Handling call
0709 12:19:12.107092 (+   290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:12.719573 (+612481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.719619 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:12.730957 (+ 11338us) log.cc:844] Serialized 52 byte log entry
0709 12:19:12.906014 (+175057us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:13.587415 (+681401us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b180047fc20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b289f455 00007f29b93873c7 00007f29b59999a7 00007f29b59a90bf
0709 12:19:13.587652 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.596400 (+  8748us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.596625 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.596710 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.615386 (+ 18676us) raft_consensus.cc:1685] finished
0709 12:19:13.615414 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.615597 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":603421,"prepare.queue_time_us":19681,"prepare.run_cpu_time_us":3936,"prepare.run_wall_time_us":13573,"raft.queue_time_us":603997,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":40749824,"thread_start_us":157449,"threads_started":2,"wal-append.queue_time_us":239843}
W20260709 12:19:13.613394 29787 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:19:13.629819 25710 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:13.602744 29791 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:13.611367 29789 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:13.634111 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78355.4us
I20260709 12:19:13.658272 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:13.659313 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.659574 25676 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:13.660349 29780 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:13.660212 29800 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:13.661223 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2355) took 1563 ms. Trace:
I20260709 12:19:13.661602 25781 rpcz_store.cc:276] 0709 12:19:12.097357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.097532 (+   175us) service_pool.cc:224] Handling call
0709 12:19:12.099594 (+  2062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.718253 (+618659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.718273 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.723729 (+  5456us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.892466 (+168737us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:12.892503 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.892692 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.600767 (+708075us) spinlock_profiling.cc:243] Waited 6.67 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:13.634854 (+ 34087us) raft_consensus.cc:1685] finished
0709 12:19:13.634873 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.661162 (+ 26289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":618205,"raft.queue_time_us":626553,"spinlock_wait_cycles":21875968,"thread_start_us":104306,"threads_started":1,"wal-append.queue_time_us":145480}
I20260709 12:19:13.660889 29689 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:19:13.669026 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:13.669593 25707 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:13.672640 29740 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.841s	user 0.009s	sys 0.000s
I20260709 12:19:13.673415 29740 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.675711 29683 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:13.677174 25711 raft_consensus.cc:2368] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:13.672523 25712 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
W20260709 12:19:13.681535 29619 consensus_meta.cc:287] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.001s
I20260709 12:19:13.683993 29740 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:13.687601 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:13.688789 29455 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:13.689344 29455 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:13.690274 29455 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:13.694687 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.696384 25713 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:13.709333 25707 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
W20260709 12:19:13.716531 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58332.3us
I20260709 12:19:13.718343 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:13.718915 25778 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:13.719858 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:13.721393 25603 connection.cc:256] RPC call timeout handler was delayed by -1.0305063810001229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
I20260709 12:19:13.728695 25603 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:13.736902 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:13.736889 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.749025 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:13.749907 25779 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:13.620262 29742 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:13.756201 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 370220us
I20260709 12:19:13.720920 29740 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.764557 29698 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:13.790854 29698 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.790737 29740 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.793164 29698 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:13.793823 29740 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.716218 29798 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:13.794857 29798 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.775395 29444 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.736284 29656 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.797567 29656 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.721024 29619 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:13.780426 29806 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.798583 29619 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.798822 29806 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.801370 29619 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.802129 29806 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:13.781750 29683 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.804175 29683 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.807993 29656 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.808786 29656 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:13.809824 29656 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:13.810840 29656 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:13.813215 29484 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.002s
I20260709 12:19:13.813815 26337 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:13.787518 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.814471 29806 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:13.815272 29806 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:13.816012 29806 consensus_queue.cc:1243] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:13.762550 29459 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:13.779496 29805 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:13.816907 29805 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.819169 29805 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:13.812347 29683 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.783159 29742 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.819803 29805 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:13.822070 29767 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:13.735764 29510 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:13.822999 29510 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:13.823524 29510 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:13.751771 29790 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.832422 29791 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:13.832901 29613 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:13.839000 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:13.839908 25632 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:13.841282 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:13.843938 29792 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.844570 29792 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.802841 29722 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:13.815471 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:13.850342 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:13.851106 25782 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:13.851282 29722 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:13.797767 29798 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:13.822475 29767 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:13.853438 29767 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.788523 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.854480 29767 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:13.854866 25707 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:13.855084 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:13.855990 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.856796 25707 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:13.857643 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:13.858421 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:13.858755 26307 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:13.859817 29699 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:13.856904 29722 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:13.860183 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:13.863045 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:13.848711 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.875947 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:13.858080 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.877496 25707 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:13.866330 25778 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:13.883504 25785 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:13.885380 25782 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:13.885612 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:13.887863 29806 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:13.889415 29459 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:13.889863 29459 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:13.890228 29459 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:13.884589 29375 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.866101 29711 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.891140 29375 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:13.893018 29711 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:13.867583 29414 consensus_meta.cc:287] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.000s
I20260709 12:19:13.894860 29414 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:13.895411 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.895859 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:13.896127 25708 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:13.896526 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:13.897305 29798 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:13.897461 29791 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:13.897912 29778 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:13.898065 29791 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.899313 25704 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:13.900281 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.905457 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:13.906440 25707 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:13.884696 29484 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:19:13.895459 29782 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.001s
I20260709 12:19:13.917511 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 664) took 1911 ms. Trace:
I20260709 12:19:13.942404 25783 rpcz_store.cc:276] 0709 12:19:12.006464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.029549 (+ 23085us) service_pool.cc:224] Handling call
0709 12:19:12.029903 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.725073 (+695170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.725100 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:12.725140 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:12.725167 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.917236 (+1192069us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54000870d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:13.917294 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.917444 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":426874,"raft.queue_time_us":402416,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":47847,"spinlock_wait_cycles":680515968}
I20260709 12:19:13.920023 25709 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:19:13.948101 25709 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:13.931358 26336 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:19:13.931054 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 844) took 1029 ms. Trace:
I20260709 12:19:13.950806 26335 rpcz_store.cc:276] 0709 12:19:12.901435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.901662 (+   227us) service_pool.cc:224] Handling call
0709 12:19:13.930873 (+1029211us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b54006caf54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:13.930997 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198137216}
I20260709 12:19:13.898643 29791 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:13.899971 29778 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.897768 29798 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.959230 25709 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:13.960780 29799 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:13.961711 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:13.962445 29779 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:13.962862 26339 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:19:13.963025 29779 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:13.964387 29779 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:13.966151 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:13.966349 29779 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:13.969082 26338 pending_rounds.cc:85] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 6
I20260709 12:19:13.969432 26338 pending_rounds.cc:107] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:19:13.969679 26338 raft_consensus.cc:2889] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:13.982335 29798 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:13.916654 29698 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:14.014117 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.916471 29484 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.023857 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:13.982474 29778 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:13.981194 29796 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:14.030164 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:13.983287 29792 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:14.026607 29484 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:14.027587 29680 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:13.956944 29782 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.042192 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:14.049110 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:14.040932 29806 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:19:14.039733 29646 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.025744 29698 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.030830 29810 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:14.032811 29722 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 12:19:14.114173 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 377351us
I20260709 12:19:14.115983 25708 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:19:14.116596 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.116825 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.127188 25785 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:19:14.119920 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.127332 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:14.046751 29805 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:19:14.041860 29414 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:14.065955 29783 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.502s	user 0.003s	sys 0.008s
I20260709 12:19:14.137042 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.137198 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.128015 25783 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:14.144944 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:14.147455 29809 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:19:14.148087 29782 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.137795 29619 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:14.149106 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:14.148983 29619 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.152470 29805 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:14.189241 25634 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:19:14.156860 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:14.159361 29701 consensus_meta.cc:287] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.888s	user 0.010s	sys 0.002s
I20260709 12:19:14.138937 29808 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:14.139881 29593 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.125373 29711 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:14.171164 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.203004 25785 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:14.199234 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.204234 25785 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:19:14.204576 25785 pending_rounds.cc:85] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 5
I20260709 12:19:14.204797 25785 pending_rounds.cc:107] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:19:14.205049 25785 raft_consensus.cc:2889] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:14.176347 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:14.207345 25704 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:14.176002 29783 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:14.185048 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 321454us
I20260709 12:19:14.186858 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:14.150738 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:14.118731 29792 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:14.194387 26309 leader_election.cc:400] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
I20260709 12:19:14.197119 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.213645 25632 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:14.197762 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:14.126542 29810 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:14.199707 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:14.200047 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:14.201591 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.208395 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:14.208555 25779 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:14.144572 29414 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.206351 25785 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:14.203598 25638 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:14.126327 29646 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.220813 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:14.166383 29375 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:14.204734 29455 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.010s	sys 0.034s
I20260709 12:19:14.241541 25704 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:19:14.246985 29656 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:19:14.232718 29352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.030s	sys 0.056s
I20260709 12:19:14.251489 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:14.364519 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 113130us
I20260709 12:19:14.253942 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:14.261055 29780 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:14.262571 29692 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:14.202299 29593 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.275986 29698 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:14.200662 29701 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.428320 29701 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:14.429103 29701 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.429700 29701 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.278695 29619 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.280324 29759 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:14.284759 29646 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:14.395924 29510 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.007s
I20260709 12:19:14.286862 25787 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:19:14.195585 29782 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 12:19:14.222452 29792 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.241295 29740 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:14.325181 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.221678 29805 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:14.193898 29642 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 12:19:14.356668 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63307.4us
I20260709 12:19:14.218739 29414 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:14.293193 29455 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.011s	sys 0.035s
I20260709 12:19:14.403857 29798 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.466555 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2358) took 1632 ms. Trace:
I20260709 12:19:14.358161 25706 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:14.467257 25786 rpcz_store.cc:276] 0709 12:19:12.833739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.835876 (+  2137us) service_pool.cc:224] Handling call
0709 12:19:12.836297 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.836741 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.836781 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.860575 (+1023794us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b2c0007afc8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:19:13.860944 (+   369us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.329259 (+468315us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:14.329314 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.329424 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.466141 (+136717us) raft_consensus.cc:1685] finished
0709 12:19:14.466165 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.466454 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40580,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132,"spinlock_wait_cycles":13102976,"thread_start_us":467870,"threads_started":1,"wal-append.queue_time_us":603322}
I20260709 12:19:14.469604 26340 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:14.251708 29459 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.368367 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:14.218771 29810 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:14.410967 29375 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:14.423210 25785 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:19:14.424189 29593 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:14.213689 29711 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.471781 29619 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.472162 29459 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:14.472420 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:14.472702 25783 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:14.473398 29711 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.432199 25710 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:14.433014 29783 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.337456 29690 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:14.336261 29613 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.475394 29792 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.442672 29646 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:14.476738 29646 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.477294 29646 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:14.479604 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:14.482410 26341 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:14.407022 29352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.030s	sys 0.057s
I20260709 12:19:14.418545 29484 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:14.490334 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 122076us
I20260709 12:19:14.452258 29740 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:14.453547 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.257911 29791 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.447253 29759 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.475299 29613 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:14.361310 29315 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.035s	user 0.007s	sys 0.012s
I20260709 12:19:14.271507 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:14.489135 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:14.483911 29742 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.518750 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:14.520365 26341 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:14.421208 29780 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.521368 26336 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:14.510869 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:14.514158 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:14.523867 25704 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:14.523960 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 395190us
W20260709 12:19:14.524716 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 320408us
I20260709 12:19:14.527212 25634 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:14.513697 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:14.512578 29455 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.532967 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:14.533123 29455 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.534142 29780 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.526849 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:14.535399 29455 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:14.536995 29414 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:14.539103 25706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2454) took 1966 ms (client timeout 1786 ms). Trace:
I20260709 12:19:14.539458 26341 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:14.535354 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:14.534328 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:14.542263 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.543154 25634 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
W20260709 12:19:14.539582 25706 rpcz_store.cc:269] 0709 12:19:12.573138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.573742 (+   604us) service_pool.cc:224] Handling call
0709 12:19:14.538909 (+1965167us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400073ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:14.539030 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3926007040}
I20260709 12:19:14.547211 25782 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:19:14.542097 26308 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:14.472437 29619 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:14.549886 25711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.004s	sys 0.036s
W20260709 12:19:14.553138 25711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.005s	sys 0.036s
I20260709 12:19:14.514976 29711 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:14.517417 29782 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:14.551245 29778 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 12:19:14.554170 29692 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:14.549729 29722 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.549328 29747 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:14.599758 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 53994.4us
I20260709 12:19:14.488008 29510 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:14.598199 26338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.031s	sys 0.032s
W20260709 12:19:14.603060 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.337903 29767 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.210s
W20260709 12:19:14.605996 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50007.1us
W20260709 12:19:14.612226 26338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.032s	sys 0.032s
I20260709 12:19:14.550837 29663 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:14.611471 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.618046 29663 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.620298 29663 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:14.555747 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:14.531101 29742 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:14.599263 29090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.014s	sys 0.035s
W20260709 12:19:14.622087 29090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.015s	sys 0.035s
I20260709 12:19:14.617151 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:14.607854 25601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.017s	sys 0.038s
I20260709 12:19:14.623345 25784 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:14.623733 25601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.017s	sys 0.038s
I20260709 12:19:14.625214 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.531553 29315 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.628450 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:14.629551 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.554237 29806 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:19:14.599908 29375 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:19:14.556340 29459 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 12:19:14.626300 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 577231us
I20260709 12:19:14.674067 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2534) took 1116 ms. Trace:
W20260709 12:19:14.615312 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 363228us
I20260709 12:19:14.667683 25780 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:14.670132 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2538) took 1110 ms. Trace:
I20260709 12:19:14.697639 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1905) took 1315 ms. Trace:
I20260709 12:19:14.693164 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:14.672263 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.665819 25778 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 12:19:14.666823 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 111175us
I20260709 12:19:14.720824 25783 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:14.718565 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:14.603655 29692 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.607966 29820 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:14.718367 25629 rpcz_store.cc:276] 0709 12:19:13.381948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.382238 (+   290us) service_pool.cc:224] Handling call
0709 12:19:13.382614 (+   376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:13.615984 (+233370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.616033 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:13.623374 (+  7341us) log.cc:844] Serialized 52 byte log entry
0709 12:19:14.202957 (+579583us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:14.246645 (+ 43688us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.247988 (+  1343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.248124 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.697324 (+449200us) raft_consensus.cc:1685] finished
0709 12:19:14.697356 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.697557 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1867,"prepare.queue_time_us":42886,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":207258,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":155,"thread_start_us":716751,"threads_started":2,"wal-append.queue_time_us":918147}
I20260709 12:19:14.642336 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.729836 25634 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:14.731235 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:14.651953 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:14.602917 29654 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:14.718077 25707 rpcz_store.cc:276] 0709 12:19:13.559307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.917149 (+357842us) service_pool.cc:224] Handling call
0709 12:19:13.933081 (+ 15932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.933415 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.933445 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.985990 (+ 52545us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.443896 (+457906us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:14.443942 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.444022 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.669880 (+225858us) raft_consensus.cc:1685] finished
0709 12:19:14.669909 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.670060 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":475,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":26809,"thread_start_us":412715,"threads_started":1,"wal-append.queue_time_us":592917}
W20260709 12:19:14.740603 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 88735.5us
I20260709 12:19:14.745201 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:14.611344 29822 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:14.650467 26337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 704) took 1683 ms (client timeout 1637 ms). Trace:
I20260709 12:19:14.599251 29414 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:14.604002 29808 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:14.662240 29509 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:14.615051 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 361161us
I20260709 12:19:14.762015 25783 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:14.607285 29782 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:14.681104 29818 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:19:14.669519 29824 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:19:14.746138 25629 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:14.681689 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:14.630532 29767 consensus_queue.cc:1243] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:19:14.747619 26337 rpcz_store.cc:269] 0709 12:19:12.967285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.001624 (+ 34339us) service_pool.cc:224] Handling call
0709 12:19:14.650251 (+1648627us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54000a37d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:14.650389 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864948096}
I20260709 12:19:14.806543 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.671108 29834 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:14.695469 29785 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:19:14.725018 29680 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.002s
I20260709 12:19:14.671288 29613 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.689038 29619 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:14.717578 25711 rpcz_store.cc:276] 0709 12:19:13.558110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.678944 (+120834us) service_pool.cc:224] Handling call
0709 12:19:13.679474 (+   530us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:13.679981 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.680021 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:13.939602 (+259581us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b2c000127c8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:19:13.958255 (+ 18653us) log.cc:844] Serialized 52 byte log entry
0709 12:19:14.553571 (+595316us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:14.609561 (+ 55990us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.609784 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.609860 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.673750 (+ 63890us) raft_consensus.cc:1685] finished
0709 12:19:14.673860 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.674006 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46444,"prepare.run_wall_time_us":560,"raft.queue_time_us":3767,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":14042,"spinlock_wait_cycles":46197504,"thread_start_us":526237,"threads_started":1,"wal-append.queue_time_us":670834}
I20260709 12:19:14.680310 29315 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:14.610739 29699 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 12:19:14.608244 29790 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.138s
I20260709 12:19:14.727627 29692 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:14.770457 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.770797 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:14.609454 29778 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.607098 29711 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.810576 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:14.811749 29839 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:14.672099 29646 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.761883 29509 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.824600 25629 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:14.615633 29722 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.828925 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.745254 29737 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:19:14.767436 29840 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:19:14.650693 29740 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.863619 25634 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
W20260709 12:19:14.883330 25524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.013s	sys 0.007s
W20260709 12:19:14.886533 25524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.013s	sys 0.007s
I20260709 12:19:14.864485 29680 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:14.757516 29654 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.665339 29780 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:19:14.868381 25638 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:14.758083 29820 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:14.869076 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 129287us
I20260709 12:19:14.822695 29792 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:14.815271 29785 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:14.805825 29414 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:19:14.879899 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.834326 29697 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:14.878968 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 143398us
I20260709 12:19:14.910386 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:14.884074 26343 raft_consensus.cc:2368] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9.
I20260709 12:19:14.882064 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:14.912600 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:14.886750 25712 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:14.893050 29680 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.870144 29791 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:14.902031 29790 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.765164 29822 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:14.891716 25708 raft_consensus.cc:2368] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:19:14.881254 25637 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:14.881119 29832 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:19:14.937783 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.861403 29613 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 12:19:14.900612 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 177805us
I20260709 12:19:14.881695 29509 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:14.865617 29090 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:14.870067 29459 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.903748 29484 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:14.916143 29841 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.926784 29510 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:14.934986 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:14.939133 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.946647 25634 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:14.978639 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.925784 29848 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.980508 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 668) took 1262 ms. Trace:
I20260709 12:19:14.982080 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.929080 29840 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.879068 29808 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:14.938947 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.951733 29834 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:14.932845 29680 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:14.941959 29806 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:14.942319 29619 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:14.974467 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:14.988261 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:15.008104 25785 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:19:14.988866 26341 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:19:15.008764 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:14.989209 25629 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:14.980957 25787 rpcz_store.cc:276] 0709 12:19:13.718232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.718679 (+   447us) service_pool.cc:224] Handling call
0709 12:19:13.718970 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.446429 (+727459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.980190 (+533761us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b540030af54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:14.980445 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305186688}
I20260709 12:19:14.919004 29697 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.000687 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:15.031085 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:15.001174 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:15.032069 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:15.001874 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66991.8us
I20260709 12:19:14.951180 29850 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:14.953940 29737 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:15.035949 25638 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:19:14.941183 29790 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:15.034123 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:15.038995 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:15.045025 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:14.995585 29848 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.046734 29861 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:15.047220 29861 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.047597 29861 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:15.048744 29848 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.049052 29820 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:15.043315 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.951614 29090 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.051342 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:15.046239 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:15.053236 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:15.056325 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45888.9us
I20260709 12:19:15.058533 25637 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:15.059518 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:15.060959 25785 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:14.958664 29459 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.005273 29806 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:15.064157 29375 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:15.064858 29654 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.063375 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:15.006826 29619 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:15.007432 29782 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:15.083071 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51098.5us
I20260709 12:19:15.090919 25638 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:15.087756 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:15.065593 29855 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.093808 25705 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:14.953723 29785 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.051178 29853 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.974004 29841 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:15.056106 29851 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.078615 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:15.030627 29832 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:19:15.108840 25631 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:19:15.112011 25631 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:15.077036 29778 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:15.094853 29828 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:19:14.999639 29840 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.065372 29613 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:15.087000 29820 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:15.098843 29619 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:15.095717 29646 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.177328 29722 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.098384 29850 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:15.099455 29810 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.000s
I20260709 12:19:15.105607 29315 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:19:15.185006 29782 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:15.191771 29613 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.193140 29785 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:15.220983 25524 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.929s	user 0.015s	sys 0.007s
I20260709 12:19:15.202701 29767 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.202010 29740 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:15.209936 29842 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 12:19:15.203645 29850 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:15.235637 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.233299 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 287030us
I20260709 12:19:15.236274 29740 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:15.237227 29838 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:15.238027 29656 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:15.238284 29838 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:15.238646 29740 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.239187 29722 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:15.114738 29855 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.250839 29722 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:15.251971 29722 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:15.253646 29722 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:15.213033 29613 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:15.243536 29843 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
I20260709 12:19:15.235773 29767 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.244634 29459 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.240608 29711 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.191722 29834 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.259864 29841 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.260531 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:15.260788 29767 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:15.260982 29790 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:15.261229 25786 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:15.262149 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:15.265215 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:15.265753 25786 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:15.253913 29855 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.117203 29853 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.270792 29841 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:15.271483 29841 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:15.273291 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:15.274060 25708 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
W20260709 12:19:15.274988 25710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2539) took 1715 ms (client timeout 1733 ms). Trace:
W20260709 12:19:15.275429 25710 rpcz_store.cc:269] 0709 12:19:13.559574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.659064 (+ 99490us) service_pool.cc:224] Handling call
0709 12:19:15.274788 (+1615724us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b5400104354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:15.274929 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771748608}
I20260709 12:19:15.276867 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:15.276913 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 672) took 1433 ms. Trace:
I20260709 12:19:15.278371 25780 rpcz_store.cc:276] 0709 12:19:13.843286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.844732 (+  1446us) service_pool.cc:224] Handling call
0709 12:19:13.937738 (+ 93006us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ae9c 00007f29b286aa5b 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:19:13.937964 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.718059 (+780095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.276674 (+558615us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54001257d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:15.276850 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694378496}
I20260709 12:19:15.279618 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:15.280987 29861 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.250377 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1876) took 1116 ms. Trace:
I20260709 12:19:15.281365 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 617) took 1136 ms. Trace:
I20260709 12:19:15.244486 29840 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.282303 25704 rpcz_store.cc:276] 0709 12:19:14.144849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.363029 (+218180us) service_pool.cc:224] Handling call
0709 12:19:14.413654 (+ 50625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.524372 (+110718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.281116 (+756744us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54002ed1d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:15.281305 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252965120}
I20260709 12:19:15.278964 29619 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:15.283907 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2369) took 1169 ms. Trace:
I20260709 12:19:15.282018 25779 rpcz_store.cc:276] 0709 12:19:14.133500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.219638 (+ 86138us) service_pool.cc:224] Handling call
0709 12:19:14.285752 (+ 66114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.337732 (+ 51980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.337780 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.362948 (+ 25168us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.767063 (+404115us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:14.816135 (+ 49072us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.816777 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.817799 (+  1022us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.250168 (+432369us) raft_consensus.cc:1685] finished
0709 12:19:15.250188 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.250320 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63283,"prepare.queue_time_us":46677,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":77310,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":56651,"thread_start_us":328190,"threads_started":1,"wal-append.queue_time_us":488399}
I20260709 12:19:15.284338 25782 rpcz_store.cc:276] 0709 12:19:14.114744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.114917 (+   173us) service_pool.cc:224] Handling call
0709 12:19:14.115272 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.551767 (+436495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.283647 (+731880us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54000de3d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:15.283850 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983051136}
I20260709 12:19:15.258205 29692 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.285039 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:15.285462 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:15.285768 25779 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:15.273674 29841 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:15.288911 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.291128 29692 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.281282 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:15.179103 29851 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.281138 29509 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:17.314446 25712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2478) took 2787 ms (client timeout 1797 ms). Trace:
I20260709 12:19:17.328043 29642 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.328401 29509 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:15.197597 29806 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.328627 29642 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.329783 29806 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:17.331087 29642 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.221634 29782 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:17.335997 25631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.129s	user 0.017s	sys 0.014s
I20260709 12:19:17.336508 29782 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.328517 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 715) took 1351 ms. Trace:
W20260709 12:19:17.328269 25712 rpcz_store.cc:269] 0709 12:19:14.527360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.535152 (+  7792us) service_pool.cc:224] Handling call
0709 12:19:17.314130 (+2778978us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54003c91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:17.314315 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792595840}
I20260709 12:19:15.301327 29619 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.210371 29810 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.337246 26338 rpcz_store.cc:276] 0709 12:19:13.968308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.968466 (+   158us) service_pool.cc:224] Handling call
0709 12:19:13.968855 (+   389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:13.980537 (+ 11682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.980575 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:13.983219 (+  2644us) log.cc:844] Serialized 52 byte log entry
0709 12:19:14.169853 (+186634us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b1800328d80. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b986358f 00007f29b28a2571 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd
0709 12:19:14.624479 (+454626us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:14.817027 (+192548us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.817369 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.901906 (+ 84537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.901988 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.319285 (+417297us) raft_consensus.cc:1685] finished
0709 12:19:15.319313 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.319482 (+   169us) inbound_call.cc:177] Queueing success response
0709 12:19:17.319983 (+2000501us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b48000ce658. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29ba8b8aa1 00007f29ba8afb37 00007f29ba8af9ca 00007f29ba9a726f 00007f29ba9a6f37 00007f29ba9bb55c 00007f29ba9bb508 00007f29ba9bb488 00007f29ba9bb443 00007f29ba9ba6ec 00007f29b98e4a96
Metrics: {"mutex_wait_us":17094,"prepare.queue_time_us":624197,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":5195,"raft.queue_time_us":235617,"raft.run_cpu_time_us":24116,"raft.run_wall_time_us":2419789,"spinlock_wait_cycles":15983744,"thread_start_us":872043,"threads_started":3,"wal-append.queue_time_us":734701,"wal-append.run_cpu_time_us":15400,"wal-append.run_wall_time_us":1673417}
I20260709 12:19:17.340288 29810 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.334757 29822 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.383960 29613 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.341322 29851 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.343359 29613 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.353711 29759 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.360666 29859 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.324195 29901 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.344455 29859 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.344795 29901 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.326568 25780 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
W20260709 12:19:17.346835 25780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1894) took 2072 ms (client timeout 2124 ms). Trace:
I20260709 12:19:15.408367 29090 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:17.347205 25780 rpcz_store.cc:269] 0709 12:19:15.274725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.281048 (+  6323us) service_pool.cc:224] Handling call
0709 12:19:17.346753 (+2065705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:17.347530 29901 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:15.433349 29850 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.436116 29861 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:15.442512 29375 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:15.443555 29680 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:15.482551 29874 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:17.350310 29874 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.351398 29680 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.352514 26336 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
W20260709 12:19:15.642879 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.445484 29877 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.972s	user 0.001s	sys 0.000s
W20260709 12:19:16.839426 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.655s (SENT) [suppressed 9 similar messages]
I20260709 12:19:17.347213 29859 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:17.352387 29850 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:16.941776 29847 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:17.359280 29847 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:17.360302 29847 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.361687 29847 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.180006 29352 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.364233 29352 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.365624 29742 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.186921 29891 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:17.188040 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 483446us
W20260709 12:19:17.209894 26343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 714) took 3244 ms (client timeout 3180 ms). Trace:
I20260709 12:19:17.221772 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1885) took 2098 ms. Trace:
I20260709 12:19:17.232184 29898 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:17.376639 29898 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:17.367654 26343 rpcz_store.cc:269] 0709 12:19:13.965624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.965822 (+   198us) service_pool.cc:224] Handling call
0709 12:19:17.209621 (+3243799us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b540055a354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:17.209789 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108762368}
I20260709 12:19:17.242591 29690 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.245939 29895 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.247072 29683 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.249418 29768 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:17.258019 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.261559 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1895) took 1968 ms (client timeout 1646 ms). Trace:
I20260709 12:19:17.264598 29853 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.267962 29900 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:17.272552 29831 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 2.393s	user 0.017s	sys 0.011s
W20260709 12:19:17.309814 25601 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 2.281s	user 0.006s	sys 0.030s
W20260709 12:19:17.313372 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.336747 25631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.131s	user 0.017s	sys 0.014s
I20260709 12:19:17.392343 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 886) took 2438 ms. Trace:
W20260709 12:19:17.393580 25601 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.365s	user 0.008s	sys 0.031s
I20260709 12:19:17.392678 25631 rpcz_store.cc:276] 0709 12:19:14.954230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.113286 (+159056us) service_pool.cc:224] Handling call
0709 12:19:15.113648 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.114032 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.114068 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:15.180983 (+ 66915us) log.cc:844] Serialized 26 byte log entry
0709 12:19:17.392014 (+2211031us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:17.392049 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.392111 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.392146 (+    35us) raft_consensus.cc:1685] finished
0709 12:19:17.392156 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.392293 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92465,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7891,"thread_start_us":2186055,"threads_started":1,"wal-append.queue_time_us":168098,"wal-append.run_cpu_time_us":2294,"wal-append.run_wall_time_us":1030418}
W20260709 12:19:17.393909 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 2.38544e+06us
I20260709 12:19:17.340741 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1918) took 2569 ms. Trace:
W20260709 12:19:15.342765 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 445366us
I20260709 12:19:17.348762 29429 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:17.352562 29874 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:15.792122 29778 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.399034 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.380340 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.367774 25782 rpcz_store.cc:276] 0709 12:19:15.123506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.286437 (+162931us) service_pool.cc:224] Handling call
0709 12:19:15.298385 (+ 11948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.298817 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.298844 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:17.195916 (+1897072us) log.cc:844] Serialized 26 byte log entry
0709 12:19:17.221057 (+ 25141us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:17.221107 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.221194 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.221529 (+   335us) raft_consensus.cc:1685] finished
0709 12:19:17.221543 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.221698 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5131,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":6789,"thread_start_us":24684,"threads_started":1,"wal-append.queue_time_us":22769}
I20260709 12:19:17.385728 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:17.387965 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.375524 29891 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:17.387797 25783 rpcz_store.cc:269] 0709 12:19:15.293610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.261273 (+1967663us) service_pool.cc:211] Skipping call since client already timed out
0709 12:19:17.261433 (+   160us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:19:17.385857 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:17.389709 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 550322us
I20260709 12:19:17.394502 25635 rpcz_store.cc:276] 0709 12:19:14.771392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.771567 (+   175us) service_pool.cc:224] Handling call
0709 12:19:14.771970 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:14.898820 (+126850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.898869 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:14.914833 (+ 15964us) log.cc:844] Serialized 52 byte log entry
0709 12:19:14.983406 (+ 68573us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:14.983491 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.340264 (+2356773us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003011d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:17.340332 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.340388 (+    56us) raft_consensus.cc:1685] finished
0709 12:19:17.340403 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.340675 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47794,"raft.queue_time_us":35598,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7585,"spinlock_wait_cycles":290387328,"thread_start_us":67840,"threads_started":1,"wal-append.queue_time_us":123977,"wal-append.run_cpu_time_us":1973,"wal-append.run_wall_time_us":1213607}
I20260709 12:19:17.399926 26337 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:17.402060 26340 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:17.388171 26343 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:19:17.402786 26343 pending_rounds.cc:85] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 6
I20260709 12:19:17.403152 26343 pending_rounds.cc:107] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:19:17.403457 26343 raft_consensus.cc:2889] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:17.403851 26343 pending_rounds.cc:107] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:19:17.404119 26343 raft_consensus.cc:2889] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:17.400866 26344 raft_consensus.cc:2368] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:17.408813 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:17.411614 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.412155 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.389961 29683 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.415079 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.415673 26339 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:17.391445 29768 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.416592 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.417276 26344 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:17.417287 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.418989 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:17.419495 26335 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:17.411480 29742 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:17.420594 29742 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.387457 29895 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.421322 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:17.422479 29375 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:17.423331 29895 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.423931 29895 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:17.416824 29901 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:17.405563 29900 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.399334 29429 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:17.429890 29901 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.430253 29429 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.431869 29429 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:17.432433 29901 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.424957 29806 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.436173 29806 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.424500 29891 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:17.439307 29742 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.440369 29768 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.386600 29690 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.440801 29613 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:17.441340 29851 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.441565 29613 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:17.443065 25676 connection.cc:256] RPC call timeout handler was delayed by -1.1478517079999619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 12:19:17.443523 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0380517080000118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
I20260709 12:19:17.424568 29895 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:17.406847 29778 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.428027 29810 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.452526 29690 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:17.445161 29796 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:19:17.454785 29823 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:17.455767 29823 consensus_queue.cc:1243] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:17.428166 29900 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:17.456236 29895 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:17.410162 29792 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.429105 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.405982 29352 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.441928 29851 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.452499 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:17.421135 29090 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:17.416589 29861 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.448807 26335 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:17.457104 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1888) took 2332 ms. Trace:
W20260709 12:19:17.443908 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0260503629999675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
I20260709 12:19:17.456264 29796 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:17.449798 29806 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.466089 25780 rpcz_store.cc:276] 0709 12:19:15.124357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.348210 (+2223853us) service_pool.cc:224] Handling call
0709 12:19:17.348619 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:17.353269 (+  4650us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:19:17.353306 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:17.353951 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 12:19:17.388560 (+ 34609us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:19:17.388603 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.388676 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.456872 (+ 68196us) raft_consensus.cc:1685] finished
0709 12:19:17.456897 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.457042 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3951,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":32792,"threads_started":1,"wal-append.queue_time_us":96465}
I20260709 12:19:17.467612 29619 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.468113 29861 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.468324 29090 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.468428 29619 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.443656 29850 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.470815 29850 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.453858 29613 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.444326 29742 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.467681 29832 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:17.443888 29759 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.473932 25601 leader_election.cc:400] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9.
I20260709 12:19:17.471973 29851 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.475975 29850 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.466086 29792 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.467303 29796 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:17.478093 29782 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.495512 29895 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.475980 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.451817 29898 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.497591 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.463617 29810 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.498338 26339 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:17.499341 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.500005 29810 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.486320 29778 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.500438 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:17.501030 26339 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.478641 29642 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.501907 29642 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.503262 29810 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:17.504206 29642 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.504424 29847 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:17.504948 29847 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:17.505919 29847 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.506675 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:17.507452 29847 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:17.507824 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:17.508695 25712 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:19:17.509567 29352 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:17.509989 29352 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.510339 29352 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:17.510695 29902 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.511270 29902 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.513517 29902 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.514034 29902 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.481669 29806 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:19:17.488175 29792 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.485422 29796 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.482658 29742 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.483372 29759 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.521297 29806 consensus_queue.cc:1243] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:17.521538 29823 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.523242 25782 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:17.523352 29759 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.523609 29810 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:17.495263 25785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.430s	user 0.011s	sys 0.031s
I20260709 12:19:17.497407 29832 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:17.524175 29810 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.524698 29832 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:17.525102 29832 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:17.526046 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 997) took 2568 ms. Trace:
I20260709 12:19:17.470412 29861 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.526664 29823 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.522809 29414 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:17.527269 29796 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:17.527588 29414 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:17.530390 29850 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.535239 29850 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.535418 29414 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.536636 29823 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.537808 29850 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.537957 29742 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.539624 29759 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.540143 29759 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.542119 29759 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.516790 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.518821 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:17.480962 29090 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.524557 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:17.549814 25782 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:17.506896 25602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.381s	user 0.011s	sys 0.017s
I20260709 12:19:17.528219 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2375) took 2522 ms. Trace:
I20260709 12:19:17.550961 25778 rpcz_store.cc:276] 0709 12:19:15.006099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.006460 (+   361us) service_pool.cc:224] Handling call
0709 12:19:15.006979 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.007332 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.007361 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:15.040082 (+ 32721us) log.cc:844] Serialized 26 byte log entry
0709 12:19:15.202694 (+162612us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:15.202739 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.202831 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.527813 (+2324982us) raft_consensus.cc:1685] finished
0709 12:19:17.528021 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.528161 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26891,"raft.queue_time_us":23066,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":14226,"thread_start_us":113521,"threads_started":1,"wal-append.queue_time_us":245528,"wal-append.run_cpu_time_us":2236,"wal-append.run_wall_time_us":1076309}
I20260709 12:19:17.527976 26336 rpcz_store.cc:276] 0709 12:19:14.957338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.957494 (+   156us) service_pool.cc:224] Handling call
0709 12:19:14.957934 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:17.353337 (+2395403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.525707 (+172370us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54006df454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:17.525974 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5466462080}
I20260709 12:19:17.487032 29619 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:17.531421 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.504779 29642 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.516463 25708 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:17.525853 29832 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:17.542825 25785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.478s	user 0.012s	sys 0.031s
I20260709 12:19:17.544486 25780 raft_consensus.cc:2410] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:19:17.554052 25602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.429s	user 0.012s	sys 0.017s
I20260709 12:19:17.561419 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:17.562538 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.563149 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2380) took 2509 ms. Trace:
I20260709 12:19:17.563503 25785 rpcz_store.cc:276] 0709 12:19:15.053701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.064231 (+ 10530us) service_pool.cc:224] Handling call
0709 12:19:15.064551 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.064778 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:15.065087 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 12:19:17.560600 (+2495513us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:17.560655 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:17.562904 (+  2249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.562981 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.563088 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124,"prepare.queue_time_us":10504,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4449,"thread_start_us":2495011,"threads_started":1,"wal-append.queue_time_us":365552,"wal-append.run_cpu_time_us":1297,"wal-append.run_wall_time_us":1002180}
I20260709 12:19:17.565176 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.565814 25785 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:17.563151 25780 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:17.568738 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.569480 25780 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:17.572008 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:17.578922 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.581873 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.583438 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.584231 25785 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:17.588263 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:17.589053 25785 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.593818 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.594655 25778 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:17.514474 29902 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.598618 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:17.496026 29782 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.552204 29861 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:17.599964 26336 raft_consensus.cc:2376] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10. Re-sending same reply.
I20260709 12:19:17.602717 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2378) took 2595 ms. Trace:
I20260709 12:19:17.558118 29613 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.641036 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.646009 26338 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:17.646916 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.581703 29429 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.642525 25787 rpcz_store.cc:276] 0709 12:19:15.007062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.009719 (+  2657us) service_pool.cc:224] Handling call
0709 12:19:15.010041 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:15.042322 (+ 32281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.042355 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:15.058318 (+ 15963us) log.cc:844] Serialized 52 byte log entry
0709 12:19:15.213268 (+154950us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:17.511905 (+2298637us) log.cc:844] Serialized 14 byte log entry
0709 12:19:17.602277 (+ 90372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.602390 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.602452 (+    62us) raft_consensus.cc:1685] finished
0709 12:19:17.602469 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.602644 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23485,"prepare.queue_time_us":2300318,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":59054,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":1870,"thread_start_us":185724,"threads_started":2,"wal-append.queue_time_us":232843,"wal-append.run_cpu_time_us":2077,"wal-append.run_wall_time_us":1080735}
I20260709 12:19:17.648929 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.558458 29851 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:17.653076 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.542176 29352 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.656737 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.594148 29894 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:17.584780 29642 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.666123 26336 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:17.560158 29806 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:19:17.645012 29832 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.671700 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.664337 26339 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:17.670218 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:17.673084 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.694876 25710 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:17.648540 29861 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.697546 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:17.699366 26339 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:17.700397 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:17.700984 26339 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.645408 29810 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.701085 29782 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.698195 29861 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.685170 29805 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.670074 29642 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.716903 26339 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:17.714996 29805 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.719373 26339 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:19:17.719331 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:17.720410 29805 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.652487 29613 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.721274 25708 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:17.567215 29792 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:19:17.703269 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:17.705543 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 118976us
I20260709 12:19:17.723076 29891 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:17.723596 29891 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:17.723663 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:17.724532 29891 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.725404 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:17.725827 29891 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:17.667557 29894 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.665390 29429 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:17.730590 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.722095 26339 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:19:17.739318 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.722375 29837 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Append to log took a long time: real 2.843s	user 0.009s	sys 0.025s
I20260709 12:19:17.741031 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:17.729319 29895 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.723088 26338 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:17.743599 26338 pending_rounds.cc:85] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 6
I20260709 12:19:17.732990 29798 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:17.726537 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:17.746733 26338 pending_rounds.cc:107] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:19:17.747310 26338 raft_consensus.cc:2889] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:17.749781 25785 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:17.741770 26341 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:17.698354 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:17.752080 25710 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:17.753671 25782 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:17.670621 29902 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:17.755542 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.756049 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.667119 29851 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.722757 29613 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.762966 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:17.743647 29683 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:17.735720 29352 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.755266 29805 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:17.728677 29894 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.771850 26341 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:17.774394 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:17.775686 29792 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:17.776384 29792 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:17.767470 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:17.769412 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:17.777441 25631 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:17.754858 29759 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:17.766144 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:17.778209 29759 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.760790 29823 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:17.778633 29759 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:17.778934 29823 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.779340 29823 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:17.783891 29858 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 2.544s	user 0.003s	sys 0.020s
I20260709 12:19:17.784446 25708 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:17.786170 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 1917) took 3015 ms. Trace:
I20260709 12:19:17.786777 29823 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.787053 25633 rpcz_store.cc:276] 0709 12:19:14.770669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.770804 (+   135us) service_pool.cc:224] Handling call
0709 12:19:14.771150 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:14.807335 (+ 36185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.807386 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:14.811553 (+  4167us) log.cc:844] Serialized 52 byte log entry
0709 12:19:14.917242 (+105689us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:19:14.918861 (+  1619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.918970 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.783019 (+2864049us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:17.785818 (+  2799us) raft_consensus.cc:1685] finished
0709 12:19:17.785838 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.786101 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39936,"raft.queue_time_us":117325,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":4637,"spinlock_wait_cycles":30263680,"thread_start_us":49607,"threads_started":1,"wal-append.queue_time_us":360045}
I20260709 12:19:17.777554 29792 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.806936 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:17.807629 25630 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:17.809655 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:17.771119 29805 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.769932 29847 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.770309 29916 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.812397 29805 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:17.819139 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.820767 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:17.822121 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:17.834437 25631 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:17.843801 25633 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:17.847297 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:17.848255 25629 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:17.850147 29894 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:17.850764 29894 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.852174 29851 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.852762 29894 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.823546 29916 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.819351 29847 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:17.858207 29916 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.859699 29847 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:17.860872 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:17.864313 25634 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260709 12:19:17.864673 25634 pending_rounds.cc:85] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 8
I20260709 12:19:17.864888 25634 pending_rounds.cc:107] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 12:19:17.865109 25634 raft_consensus.cc:2889] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:17.866617 25632 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:17.868886 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:17.872498 29778 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:17.872887 29778 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.873203 29778 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:17.874606 29429 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:17.875967 29837 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:17.876062 25636 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:17.834757 29613 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.877290 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:17.876914 29613 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:17.880699 29613 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.823325 29823 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.882648 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:17.886983 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:17.882243 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:17.900493 25779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.612s	user 0.013s	sys 0.033s
W20260709 12:19:17.903688 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.888818 29874 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:17.895987 29898 raft_consensus.cc:2764] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:19:17.910871 29701 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:17.910764 29874 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.908360 29509 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:19:17.908663 29429 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:19:17.911106 29692 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:19:17.930258 29874 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.913448 29759 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:17.948176 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 137257us
W20260709 12:19:17.953451 25779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.665s	user 0.014s	sys 0.035s
I20260709 12:19:17.954582 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1886) took 2830 ms. Trace:
I20260709 12:19:17.954948 25779 rpcz_store.cc:276] 0709 12:19:15.123827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.286608 (+162781us) service_pool.cc:224] Handling call
0709 12:19:15.286943 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.287566 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.287595 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:15.288069 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:19:17.954152 (+2666083us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:17.954205 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.954290 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.954356 (+    66us) raft_consensus.cc:1685] finished
0709 12:19:17.954368 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.954522 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":87,"thread_start_us":2665609,"threads_started":1,"wal-append.queue_time_us":150743,"wal-append.run_cpu_time_us":1523,"wal-append.run_wall_time_us":1001565}
I20260709 12:19:17.950078 25631 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:17.960661 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:17.960292 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:17.963706 25636 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:17.961930 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:17.965371 25632 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.962013 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:17.966204 29509 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:17.978577 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.972465 25779 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:17.988543 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.990761 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.961416 29921 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.995388 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.051551 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.007975 25632 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:17.996210 25787 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:18.056013 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.055289 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.060596 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:18.011880 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.040822 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.061579 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.061772 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.061957 25631 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:18.042052 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.063498 25779 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:18.039781 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:18.062107 25778 raft_consensus.cc:2368] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:18.063551 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.062383 25636 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:18.065692 25780 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:19:18.066756 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.068037 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.070266 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:18.071518 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.074859 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:17.977065 29509 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:18.077306 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:18.073892 29805 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:18.092137 25779 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:18.077500 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:18.084239 25787 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:18.076125 29927 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:18.114432 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.120211 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.122473 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:18.123245 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.120654 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.119586 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.123443 25633 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:19:18.123900 25779 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:19:18.125378 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.126106 25779 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:18.054704 29921 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.127969 29778 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.129756 29921 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:18.130842 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:18.130856 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.131536 25636 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:19:18.131927 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.132226 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.133042 26337 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:18.133034 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.133816 26339 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:18.134130 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.134902 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:18.131117 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.123652 25778 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:18.130229 29778 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:18.158592 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.161661 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.186571 26341 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:18.157382 25633 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:18.162533 25782 raft_consensus.cc:2410] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:18.187335 25784 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:19:18.187662 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:18.180480 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.188732 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:18.161312 25786 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 2.895s	user 0.016s	sys 0.017s
I20260709 12:19:18.179942 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.114837 25631 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:18.189431 25786 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:18.190860 25633 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:18.189082 25632 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:18.191594 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1889) took 3067 ms (client timeout 1962 ms). Trace:
I20260709 12:19:18.178638 25779 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:19:18.192035 25786 rpcz_store.cc:269] 0709 12:19:15.124608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.264964 (+140356us) service_pool.cc:224] Handling call
0709 12:19:18.191516 (+2926552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:18.118328 25787 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:18.144022 29934 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.093s	user 0.000s	sys 0.003s
I20260709 12:19:18.189553 25601 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:18.114002 29857 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:18.200552 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.113946 29928 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:18.196326 29938 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:19:18.197095 29932 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:19:18.198156 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.198140 25636 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:18.194432 25603 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:18.139104 29927 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:18.206616 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.183244 29759 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:18.190310 29796 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:18.222678 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.223824 25784 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:18.224890 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.208735 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.186271 29921 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:18.216993 25779 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:18.217855 25632 raft_consensus.cc:2410] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:19:18.219717 25603 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:18.209622 25780 raft_consensus.cc:2376] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:18.220873 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:18.225791 25787 raft_consensus.cc:2368] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:18.226123 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.207979 29832 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:18.230365 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.233538 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.233794 25636 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:18.235735 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:18.236641 25631 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:18.220197 29920 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:18.200575 29857 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:18.240842 25603 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:18.245824 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.249017 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:18.250033 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:18.222863 29796 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:18.233065 29832 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.238303 29839 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:18.238240 29823 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.243732 29759 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:18.271008 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:18.281260 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.293706 25779 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:18.282097 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.294752 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.269773 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:18.295565 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:18.295920 25631 raft_consensus.cc:2410] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:19:18.296598 25704 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:18.281663 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.297983 25636 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:18.258764 25784 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:18.234861 29921 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:18.215790 29509 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:18.307801 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.315017 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.316038 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.316354 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:18.319197 25634 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:18.318336 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:18.320472 26335 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:18.327328 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.327627 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.328269 26341 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:18.330571 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.331321 26341 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:18.252573 29920 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:18.333542 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.254235 29832 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:18.354681 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.345505 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:18.356115 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.351729 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.247349 29916 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:19:18.262725 29805 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:18.356354 26338 raft_consensus.cc:2410] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:19:18.360435 25779 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:18.360664 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.361910 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:18.356925 26335 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:18.357364 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.346760 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:18.363641 25636 raft_consensus.cc:2376] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10. Re-sending same reply.
I20260709 12:19:18.359269 26337 raft_consensus.cc:2368] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:18.268388 29894 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:18.252345 29799 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 12:19:18.372098 26339 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:18.300315 29823 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.360175 29759 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:18.366262 29920 raft_consensus.cc:2764] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:19:18.374976 29805 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:18.389763 25751 connection.cc:256] RPC call timeout handler was delayed by -2.964166256999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
W20260709 12:19:18.390281 25751 connection.cc:256] RPC call timeout handler was delayed by -2.535292224999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:19:18.390581 25751 connection.cc:256] RPC call timeout handler was delayed by -2.4331528219997836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:19:18.390838 25751 connection.cc:256] RPC call timeout handler was delayed by -2.2814922249999654s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
W20260709 12:19:18.391111 25751 connection.cc:256] RPC call timeout handler was delayed by -2.2481922249999116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 12:19:18.391360 25751 connection.cc:256] RPC call timeout handler was delayed by -2.2184922249998635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2263s for a response
W20260709 12:19:18.391623 25751 connection.cc:256] RPC call timeout handler was delayed by -2.0559283540001161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 12:19:18.391889 25751 connection.cc:256] RPC call timeout handler was delayed by -2.0398528219998298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 12:19:18.392156 25751 connection.cc:256] RPC call timeout handler was delayed by -1.958462377999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
W20260709 12:19:18.392413 25751 connection.cc:256] RPC call timeout handler was delayed by -1.6370055479999337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
W20260709 12:19:18.392669 25751 connection.cc:256] RPC call timeout handler was delayed by -1.5916528219997872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2559s for a response
W20260709 12:19:18.392920 25751 connection.cc:256] RPC call timeout handler was delayed by -1.5007528219998676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.266s for a response
I20260709 12:19:18.350462 29874 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.395231 29796 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:18.395893 29856 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:18.396286 29856 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:18.397083 29856 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.398089 29856 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:18.402166 26341 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:18.398157 29874 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:18.402446 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.387323 29833 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 3.507s	user 0.018s	sys 0.002s
I20260709 12:19:18.404094 29805 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:18.404291 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2462) took 4679 ms. Trace:
I20260709 12:19:18.404999 29823 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:18.404667 25713 rpcz_store.cc:276] 0709 12:19:13.724748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.979604 (+254856us) service_pool.cc:224] Handling call
0709 12:19:14.021110 (+ 41506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.028723 (+  7613us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:19:14.028750 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.033752 (+  5002us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.596420 (+562668us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:14.596462 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.596543 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:18.386787 (+3790244us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:18.403971 (+ 17184us) raft_consensus.cc:1685] finished
0709 12:19:18.403991 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.404211 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3262,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5420,"spinlock_wait_cycles":3689728,"thread_start_us":441115,"threads_started":1,"wal-append.queue_time_us":826716}
I20260709 12:19:18.406025 29429 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:18.406749 29429 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:18.410172 29832 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:18.412287 29832 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:18.351483 29839 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:18.415213 29839 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:18.386982 29937 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:19:18.285449 29857 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.421640 29857 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:18.424551 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.425237 25783 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:18.387310 29759 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:18.426515 29759 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:18.427075 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.427704 25783 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:18.428881 29697 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:18.429567 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.430248 25783 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:18.435786 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.437813 25783 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:18.319453 29927 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.444658 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.446616 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.385145 29799 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:18.491531 29799 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:18.494060 29799 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:18.368531 29894 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:18.417632 29921 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:18.418949 29429 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:18.560041 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 243567us
I20260709 12:19:18.960341 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:18.961700 29962 raft_consensus.cc:2764] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:19:18.571686 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 321507us
I20260709 12:19:18.965072 25603 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:18.966212 29962 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:18.966779 29962 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.967250 29962 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:18.573235 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.574383 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.576121 25633 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:18.598176 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.615410 25524 maintenance_manager.cc:643] P 3c2fa83ead874145ba497a2c6d2a12d0: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 13.087s	user 0.263s	sys 0.110s Metrics: {"bytes_written":30638,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":120573,"dirs.run_cpu_time_us":709,"dirs.run_wall_time_us":4357,"drs_written":1,"lbm_read_time_us":521,"lbm_reads_lt_1ms":4,"lbm_write_time_us":155536,"lbm_writes_1-10_ms":4,"lbm_writes_10-100_ms":6,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":650135,"threads_started":2,"update_count":26,"wal-append.queue_time_us":814910,"wal-append.run_cpu_time_us":861,"wal-append.run_wall_time_us":1138630}
I20260709 12:19:18.664319 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2559) took 1072 ms. Trace:
I20260709 12:19:18.678799 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:18.470245 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:18.461465 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 150507us
I20260709 12:19:18.461136 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.457615 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:18.976452 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 506059us
I20260709 12:19:18.693158 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:18.977234 29891 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:18.450196 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.977766 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.467636 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.454527 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.460445 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.979383 25787 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:18.469224 25786 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:18.695185 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.449736 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.702714 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.499176 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.972076 25709 rpcz_store.cc:276] 0709 12:19:17.591525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.271531 (+680006us) service_pool.cc:224] Handling call
0709 12:19:18.271939 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.272378 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:18.272418 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.273038 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:19:18.462809 (+189771us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:18.462853 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.462925 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:18.663358 (+200433us) raft_consensus.cc:1685] finished
0709 12:19:18.663439 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.663951 (+   512us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2126,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":189117,"threads_started":1,"wal-append.queue_time_us":210559}
I20260709 12:19:18.494736 29894 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:18.986799 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:18.987248 25752 leader_election.cc:400] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:19:18.987759 25752 leader_election.cc:403] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:19:18.738554 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.991463 25525 maintenance_manager.cc:419] P 3c2fa83ead874145ba497a2c6d2a12d0: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9397 bytes on disk
I20260709 12:19:18.991925 29891 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:19:18.739318 29856 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.995113 25631 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:18.995153 29856 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:18.982355 25781 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:19:18.997457 25524 maintenance_manager.cc:643] P 3c2fa83ead874145ba497a2c6d2a12d0: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.002s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":559,"lbm_reads_lt_1ms":4}
I20260709 12:19:18.739871 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:18.760579 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:18.760835 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:18.764057 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 127681us
W20260709 12:19:18.798180 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.812389 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:18.827064 29414 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:18.834460 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:18.856323 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:18.856832 29928 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.001s	sys 0.007s
W20260709 12:19:18.864238 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.003228 25704 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:19:18.878736 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.885576 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2479) took 4357 ms. Trace:
I20260709 12:19:18.890269 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2553) took 1369 ms. Trace:
I20260709 12:19:18.893944 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1008) took 1512 ms. Trace:
I20260709 12:19:18.895928 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:18.927721 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:18.944547 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:18.973543 29921 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:19:19.005767 25705 rpcz_store.cc:276] 0709 12:19:14.527676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.535669 (+  7993us) service_pool.cc:224] Handling call
0709 12:19:14.535985 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.105902 (+569917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.884965 (+3779063us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54002251d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:18.885338 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1282371968}
I20260709 12:19:19.007984 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.008392 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.009091 26344 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:19.009560 25705 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.011018 29960 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:19.011663 29960 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.011979 25711 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:19:19.012513 29953 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:19.012979 29953 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:19.013506 29953 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:19.013962 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:19.014896 25785 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:19.014815 25634 raft_consensus.cc:2376] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8. Re-sending same reply.
I20260709 12:19:19.015280 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:19.016353 25629 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:18.976718 25784 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:18.467895 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.017627 25629 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:18.974470 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:19.018091 25780 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:18.977658 25712 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:19.018873 25636 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.006350 26342 rpcz_store.cc:276] 0709 12:19:17.381594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.382289 (+   695us) service_pool.cc:224] Handling call
0709 12:19:17.387309 (+  5020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:17.391272 (+  3963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:17.391305 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:17.391355 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:17.391390 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.893236 (+1501846us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b54004975d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:18.893328 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.893617 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8626048}
I20260709 12:19:18.982129 25710 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
W20260709 12:19:19.023108 29928 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.001s	sys 0.007s
I20260709 12:19:19.023633 29928 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:19.005950 25707 rpcz_store.cc:276] 0709 12:19:17.519488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.455750 (+936262us) service_pool.cc:224] Handling call
0709 12:19:18.456219 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.456686 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:18.456723 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.457269 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:19:18.664017 (+206748us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:18.748611 (+ 84594us) log.cc:844] Serialized 14 byte log entry
0709 12:19:18.876193 (+127582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.877940 (+  1747us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:18.888085 (+ 10145us) raft_consensus.cc:1685] finished
0709 12:19:18.888113 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.888969 (+   856us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24974,"prepare.queue_time_us":127277,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":714,"raft.queue_time_us":33577,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":1685,"thread_start_us":395918,"threads_started":3,"wal-append.queue_time_us":37131}
I20260709 12:19:19.025336 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 897) took 1219 ms. Trace:
I20260709 12:19:19.025214 29928 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.012025 29960 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:19.004791 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:19.027999 25632 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:19.008432 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 176616us
I20260709 12:19:18.451650 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:18.870909 25637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.673s	user 0.033s	sys 0.043s
I20260709 12:19:19.011121 25708 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:19.030475 29509 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:19.010126 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:18.987769 25709 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
W20260709 12:19:19.040206 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75466.4us
I20260709 12:19:19.039592 29484 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:19.032763 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.025830 25635 rpcz_store.cc:276] 0709 12:19:17.805525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.805700 (+   175us) service_pool.cc:224] Handling call
0709 12:19:17.806024 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:17.867620 (+ 61596us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:17.919145 (+ 51525us) log.cc:844] Serialized 14 byte log entry
0709 12:19:18.043692 (+124547us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:18.043784 (+    92us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:18.043862 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.024729 (+980867us) spinlock_profiling.cc:243] Waited 61.4 ms on lock 0x7b54001d6fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:19.024823 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.025110 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48690,"prepare.queue_time_us":74826,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":55452,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":101,"spinlock_wait_cycles":141118208,"thread_start_us":126488,"threads_started":2,"wal-append.queue_time_us":155404}
I20260709 12:19:19.046514 29509 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.047504 29901 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:19.048098 29901 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:19.049270 29901 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.050990 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1012) took 1623 ms. Trace:
I20260709 12:19:19.051710 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:19.056782 29484 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.051417 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:19.059245 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 53276.4us
I20260709 12:19:19.058569 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:19.070876 25637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.883s	user 0.034s	sys 0.044s
I20260709 12:19:19.067466 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.071201 25705 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.075054 29950 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:19.075729 29950 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.076187 29950 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:19.078825 25709 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:19.079470 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2256) took 4141 ms. Trace:
I20260709 12:19:19.079864 25637 rpcz_store.cc:276] 0709 12:19:14.937708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.100436 (+162728us) service_pool.cc:224] Handling call
0709 12:19:15.100963 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.101377 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.101417 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:15.102061 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 12:19:19.079003 (+3976942us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:19.079060 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.079167 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:19.079225 (+    58us) raft_consensus.cc:1685] finished
0709 12:19:19.079234 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.079388 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11,"raft.queue_time_us":102410,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3395,"thread_start_us":3890620,"threads_started":1,"wal-append.queue_time_us":314986,"wal-append.run_cpu_time_us":1897,"wal-append.run_wall_time_us":1009281}
W20260709 12:19:19.083325 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 67285.1us
I20260709 12:19:19.084580 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.061604 29856 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:19.085808 29856 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.086480 29856 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.087517 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 909) took 1128 ms. Trace:
I20260709 12:19:19.087791 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.088856 25704 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.089144 29856 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:19.093308 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.098871 29847 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:19.051401 26336 rpcz_store.cc:276] 0709 12:19:17.427285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.712852 (+285567us) service_pool.cc:224] Handling call
0709 12:19:17.713275 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:17.738042 (+ 24767us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:17.760907 (+ 22865us) log.cc:844] Serialized 14 byte log entry
0709 12:19:17.941921 (+181014us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:17.943781 (+  1860us) log.cc:844] Serialized 26 byte log entry
0709 12:19:17.944235 (+   454us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:17.944286 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.784288 (+840002us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b1800473f20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419
0709 12:19:19.050447 (+266159us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b540058ff54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:19.050517 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:19.050583 (+    66us) raft_consensus.cc:1685] finished
0709 12:19:19.050596 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.050813 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25149,"prepare.queue_time_us":22659,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":606,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":14038,"spinlock_wait_cycles":62389888,"thread_start_us":167009,"threads_started":1,"wal-append.queue_time_us":199971}
I20260709 12:19:19.082183 25712 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:19.103936 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74364.9us
I20260709 12:19:19.087970 25630 rpcz_store.cc:276] 0709 12:19:17.959443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.959582 (+   139us) service_pool.cc:224] Handling call
0709 12:19:17.966882 (+  7300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.139329 (+172447us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:18.153728 (+ 14399us) log.cc:844] Serialized 14 byte log entry
0709 12:19:18.221130 (+ 67402us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.221493 (+   363us) log.cc:844] Serialized 26 byte log entry
0709 12:19:18.221598 (+   105us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:18.221617 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.086364 (+864747us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001c66d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:19.086440 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:19.086999 (+   559us) raft_consensus.cc:1685] finished
0709 12:19:19.087104 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.087317 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14304,"prepare.queue_time_us":15153,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":14820,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":6355,"spinlock_wait_cycles":370407296,"thread_start_us":67006,"threads_started":1,"wal-append.queue_time_us":205411}
I20260709 12:19:19.108803 25708 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:19.110839 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 64794.6us
W20260709 12:19:19.121362 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48362.5us
W20260709 12:19:19.121183 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 131621us
I20260709 12:19:19.107939 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:19.120712 25629 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:19.123950 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.139701 26336 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:19.143424 29927 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:19:19.145098 26307 leader_election.cc:336] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.524s (SENT)
I20260709 12:19:19.145670 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:19.147337 25638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 887) took 4148 ms (client timeout 2067 ms). Trace:
W20260709 12:19:19.147863 25638 rpcz_store.cc:269] 0709 12:19:14.998263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.030767 (+ 32504us) service_pool.cc:224] Handling call
0709 12:19:19.146890 (+4116123us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b54001c4dd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:19.147084 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9307008}
I20260709 12:19:19.187477 25705 raft_consensus.cc:2410] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 12:19:19.199329 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 131958us
I20260709 12:19:19.204177 29888 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:19.206947 29874 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.207631 29874 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.211711 29966 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:19:19.214414 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:19.206247 29968 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:19:19.219200 29759 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.011s
I20260709 12:19:19.219733 29759 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:19.220474 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 136123us
I20260709 12:19:19.221776 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:19.221899 29953 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.223237 29484 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.224323 29953 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.226660 29943 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.227337 29943 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.228983 29943 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.239544 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.240597 25708 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:19.243746 25705 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:19:19.243629 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:19.244642 26335 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:19.246006 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.247736 25603 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:19.248857 29943 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:19.251585 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:19.252549 29943 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:19.252766 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:19.253005 29943 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:19.253474 29943 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:19.253854 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.247990 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:19.247915 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.257498 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:19.258828 29484 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:19.259936 29484 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:19.260396 29484 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:19.261441 29484 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.263191 29759 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.210290 29874 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.268354 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.270691 29968 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:19.268304 29962 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.277549 29962 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:19.253638 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59276.4us
I20260709 12:19:19.267107 29484 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:19.265467 25711 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:19.267495 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 73089.4us
I20260709 12:19:19.270044 29894 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:19.295187 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69161.4us
I20260709 12:19:19.278136 29805 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.305655 29970 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:19.312112 29921 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.312839 29921 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:19.314657 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55417.8us
I20260709 12:19:19.315856 29429 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:19.319448 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.320169 25709 raft_consensus.cc:2376] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 11. Re-sending same reply.
I20260709 12:19:19.320235 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.320498 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.321048 25712 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:19.321201 25711 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:19.322279 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.323125 25705 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.322860 29759 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:19.324581 25710 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:19:19.327308 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.327944 29792 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:19.328130 25710 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:19.313234 29509 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.329695 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.330041 29792 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:19.330356 25705 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:19.306522 29962 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:19:19.334668 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 68427.6us
I20260709 12:19:19.336141 25704 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:19.337682 25633 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:19.338997 29894 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:19.339758 26307 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.894s (SENT)
I20260709 12:19:19.341747 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.342471 25711 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:19.343158 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:19.343678 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.344122 25635 raft_consensus.cc:2410] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:19:19.344851 25712 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.345589 25636 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:19.346210 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.346992 25709 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:19.346725 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 98817.8us
I20260709 12:19:19.326210 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.308472 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:19.355660 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46901.7us
I20260709 12:19:19.351202 29792 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.361227 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.366529 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:19.367589 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59231.3us
I20260709 12:19:19.361737 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.362866 25708 raft_consensus.cc:2410] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:19:19.368326 25710 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:19.371281 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.371203 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:19.375825 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.327425 29950 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.378364 29429 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 12:19:19.380990 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54941.7us
I20260709 12:19:19.382422 29898 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:19:19.393301 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.393433 29697 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:19.394582 29429 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:19.394879 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.394999 29429 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:19.395224 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:19.396046 29429 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.396852 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.394632 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.398818 29613 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:19.398283 29898 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:19.399654 29898 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:19.399825 29697 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:19.400219 29697 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:19.400815 29898 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.401347 29697 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.402374 29898 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:19.404346 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.404953 25634 raft_consensus.cc:2410] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:19:19.405038 25636 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:19.398272 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.399029 25710 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:19.396080 25631 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
W20260709 12:19:19.406024 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 275258us
W20260709 12:19:19.412885 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.413693 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.413996 25535 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 7 to 8, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 8 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:19.420827 25535 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 4 to 7, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 7 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:19.417883 29697 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.423341 29921 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.425084 29955 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:19.426877 25676 leader_election.cc:400] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:19.427248 25676 leader_election.cc:403] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:19.427765 29613 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:19.428359 29613 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:19.429653 29613 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.430119 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.406983 29970 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:19.436785 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.437685 25710 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.438223 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.438632 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.438791 25707 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:19.439265 25710 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:19.441139 29913 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:19.436934 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61276.1us
I20260709 12:19:19.437508 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.446841 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.426793 25535 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 5, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 5 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:19.453182 29891 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:19:19.453682 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.458396 25704 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:19:19.463899 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.464672 25709 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
W20260709 12:19:19.463301 25534 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 298) took 32582 ms (client timeout 15000 ms). Trace:
I20260709 12:19:19.471582 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.472878 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.473985 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.474697 29888 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.475358 29888 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.475929 29894 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:19.476331 29894 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:19.477218 29894 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.478581 29921 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.479192 29921 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.481494 29921 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.482141 25713 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:19.484148 29429 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.484882 25710 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:19:19.486014 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 115005us
I20260709 12:19:19.487166 29970 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:19.488310 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:19.491729 29968 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:19:19.492599 25808 debug-util.cc:398] Leaking SignalData structure 0x7b0800086520 after lost signal to thread 24424
W20260709 12:19:19.493995 25808 debug-util.cc:398] Leaking SignalData structure 0x7b080038d2a0 after lost signal to thread 25570
W20260709 12:19:19.494371 25808 debug-util.cc:398] Leaking SignalData structure 0x7b080019c6a0 after lost signal to thread 25584
W20260709 12:19:19.495997 25808 debug-util.cc:398] Leaking SignalData structure 0x7b08003218a0 after lost signal to thread 25662
I20260709 12:19:19.496045 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.496959 25711 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
W20260709 12:19:19.497676 25808 debug-util.cc:398] Leaking SignalData structure 0x7b08003c2640 after lost signal to thread 25737
I20260709 12:19:19.484069 29857 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.485061 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.502816 29857 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.503538 25709 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.504225 25677 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.505209 29913 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:19.505812 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.506703 29964 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:19.508767 25707 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
W20260709 12:19:19.466674 25534 rpcz_store.cc:269] 0709 12:18:46.875463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.880104 (+  4641us) service_pool.cc:224] Handling call
0709 12:18:47.464194 (+584090us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:18:47.574033 (+109839us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5400080610. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286b640 00007f29bb742846 00007f29bb84c90e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:19.444096 (+31870063us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:09.518411 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:09.689469 (+171058us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:09.689528 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:19:09.689576 (+    48us) tablet.cc:662] Decoding operations
0709 12:19:10.281978 (+592402us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:10.314825 (+ 32847us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:10.319266 (+  4441us) tablet.cc:685] Acquiring locks for 26 operations
0709 12:19:10.490738 (+171472us) tablet.cc:701] Row locks acquired
0709 12:19:10.490793 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:19:10.491015 (+   222us) write_op.cc:270] Start()
0709 12:19:10.491183 (+   168us) write_op.cc:276] Timestamp: P: 1783599550490922 usec, L: 0
0709 12:19:10.491209 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:19:10.603689 (+112480us) log.cc:844] Serialized 10387 byte log entry
0709 12:19:11.704750 (+1101061us) op_driver.cc:464] REPLICATION: finished
0709 12:19:11.856527 (+151777us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b28850ac 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b93c5ef0
0709 12:19:11.940403 (+ 83876us) write_op.cc:301] APPLY: starting
0709 12:19:11.940507 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:11.942198 (+  1691us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:11.942239 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:11.958765 (+ 16526us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:11.959215 (+   450us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=52
0709 12:19:11.959260 (+    45us) write_op.cc:312] APPLY: finished
0709 12:19:11.959833 (+   573us) log.cc:844] Serialized 176 byte log entry
0709 12:19:11.960740 (+   907us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:11.960971 (+   231us) write_op.cc:454] Released schema lock
0709 12:19:11.961086 (+   115us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:11.961540 (+   454us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:11.962579 (+  1039us) write_op.cc:454] Released schema lock
0709 12:19:12.339966 (+377387us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598c32a 00007f29b5911f14 00007f29b5911ec3 00007f29b5911e38 00007f29b5911df3 00007f29b59110dc 00007f29b593ae16 00007f29b5922cbb 00007f29b590d39d 00007f29b5910d75
Metrics: {"mutex_wait_us":5462691,"reported_tablets":47,"spinlock_wait_cycles":1075158272,"child_traces":[["op",{"apply.queue_time_us":230211,"apply.run_cpu_time_us":6738,"apply.run_wall_time_us":21078,"num_ops":26,"prepare.queue_time_us":239683,"prepare.run_cpu_time_us":22201,"prepare.run_wall_time_us":1403114,"raft.queue_time_us":360509,"raft.run_cpu_time_us":18676,"raft.run_wall_time_us":722394,"replication_time_us":1154376,"spinlock_wait_cycles":13099776,"thread_start_us":767440,"threads_started":4,"wal-append.queue_time_us":349685,"wal-append.run_cpu_time_us":24168,"wal-append.run_wall_time_us":1990669}]]}
I20260709 12:19:19.443172 29953 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:19:19.442986 29955 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:19.513809 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 235700us
I20260709 12:19:19.518777 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:19.505276 29857 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.458595 29901 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:19.524803 26307 leader_election.cc:336] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.879s (SENT)
W20260709 12:19:19.531564 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 51596.7us
I20260709 12:19:19.532960 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:19.527360 29960 consensus_meta.cc:287] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.500s	user 0.014s	sys 0.000s
I20260709 12:19:19.543187 29960 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.545423 25706 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:19.546604 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:19.548374 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.549238 29839 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:19.551652 29759 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:19.545536 25535 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 8, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 8 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:19.558805 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.561038 25677 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.561456 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.517144 29913 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:19.453194 29509 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:19:19.561656 25535 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 4, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:19.558830 25808 debug-util.cc:398] Leaking SignalData structure 0x7b080013ef20 after lost signal to thread 25811
I20260709 12:19:19.562320 25783 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:19.563820 29953 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:19.597641 25808 debug-util.cc:398] Leaking SignalData structure 0x7b0800410e20 after lost signal to thread 26368
W20260709 12:19:19.601505 25808 debug-util.cc:398] Leaking SignalData structure 0x7b08000635e0 after lost signal to thread 26369
I20260709 12:19:19.599440 25535 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 3 to 4, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 4 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:19.589993 29955 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:19:19.601629 26307 leader_election.cc:336] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.896s (SENT)
I20260709 12:19:19.530588 29792 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:19.608059 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.609278 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:19.610654 25634 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:19.598830 29970 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:19.612885 29970 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.614149 29894 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:19.607031 29484 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:19.615207 29484 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.616339 29970 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.617486 29484 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.619304 25786 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.570928 29943 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.596972 29697 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:19.594256 29953 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:19.626341 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.631886 25781 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:19.627163 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.594615 29913 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:19.626801 25535 catalog_manager.cc:5697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 2 to 6, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 6 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:19.632978 25779 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 6.
I20260709 12:19:19.618841 29955 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.618204 29960 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.625524 29888 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:19.646327 29805 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:19.647843 25787 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:19.650606 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:19.651064 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:19.668445 25780 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:19.651928 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:19.652109 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:19.652536 25808 debug-util.cc:398] Leaking SignalData structure 0x7b0800383160 after lost signal to thread 29808
I20260709 12:19:19.654223 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:19.607216 29613 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:19:19.640584 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 210550us
I20260709 12:19:19.667599 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:19.683848 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.687484 29981 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:19.679282 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:19.683851 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:19.688270 25636 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:19.691293 25707 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:19:19.690910 25633 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:19.693795 25704 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:19.694892 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.689958 29981 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.695581 25778 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:19.668731 29697 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.697285 29981 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.697952 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:19.698472 29697 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.702798 29984 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:19.703796 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.704792 29984 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:19.672219 29613 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.720994 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:19.721879 25808 debug-util.cc:398] Leaking SignalData structure 0x7b08003b1a00 after lost signal to thread 29838
I20260709 12:19:19.690721 25631 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:19.661710 29955 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.731308 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:19.705253 29984 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:19.711982 25603 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:19.714186 29799 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.714752 29964 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:19.715746 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.715868 29857 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.716313 29968 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.718146 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:19.718704 29921 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:19.719182 29962 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:19.719309 29414 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:19.719636 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.720400 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.720927 29966 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:19.732376 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43661.4us
I20260709 12:19:19.732856 25786 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:19.732131 26338 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:19.740754 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:19.690357 29839 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:19:19.740949 29984 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:19.750844 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:19.775645 26339 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 5.
I20260709 12:19:19.730799 29759 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 12:19:19.751653 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 197740us
I20260709 12:19:19.752260 29847 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:19.754647 29953 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.756385 29943 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.765836 29799 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.786221 29414 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:19.757050 29913 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.786815 29414 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:19.788048 29799 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.787897 29414 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.789439 29414 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:19.762984 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:19.763569 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 59311.7us
I20260709 12:19:19.688927 29950 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:19.751448 25631 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:19.765093 26341 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:19.767318 25778 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:19.768002 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:19.768848 25780 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:19.769639 25638 raft_consensus.cc:2393] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:19.733831 29613 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:19.770257 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:19.750360 26335 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:19.751421 25779 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:19.769869 29857 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.774629 29968 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.760290 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:19.791246 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.778453 29966 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.797853 29792 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.860213 29968 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:19.804440 25707 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:19:19.782949 29891 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:19.882817 29891 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.885352 29891 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:19.812402 26342 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:19.768390 29964 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.887804 25784 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:19.888967 29964 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.822433 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:19.824106 25634 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:19:19.835765 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44601.8us
I20260709 12:19:19.836431 29979 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:19.833422 25630 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:19.837611 29966 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:19.825003 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:19.894551 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:19.895316 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 145322us
I20260709 12:19:19.895653 26337 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:19.897517 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 75176us
W20260709 12:19:19.898145 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 73235.9us
I20260709 12:19:19.900055 25629 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:19.898801 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:19.906606 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:19.913954 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:19.915792 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.874143 29953 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:19.918196 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:19.919909 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:19.918846 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:19.909444 29898 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:19.925261 29898 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.925920 29839 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:19.926347 29759 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:19.944612 29759 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:19.945668 29759 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.947152 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.948118 25779 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:19.948899 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:19.950230 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.950745 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.951177 25786 raft_consensus.cc:2410] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:19:19.951578 25779 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:19.952383 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:19.952997 25784 raft_consensus.cc:2376] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11. Re-sending same reply.
I20260709 12:19:19.905654 29950 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:19.964128 29976 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.964860 29976 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:19.965148 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.965255 29955 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:19.965837 25779 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:19.966969 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.966981 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.969141 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.927737 29996 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:19:19.929488 29994 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:19:19.894515 29979 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:19.941857 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:19.942010 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 43702.8us
I20260709 12:19:19.968281 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.973659 29976 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:19.910717 29805 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:19.988329 25779 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:19.977705 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.988546 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:19.986194 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.977250 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:19.979394 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:19.995419 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:19.982761 25676 connection.cc:256] RPC call timeout handler was delayed by -2.3461431020000418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2605s for a response
I20260709 12:19:19.996349 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.997812 26307 leader_election.cc:400] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:19.998304 26307 leader_election.cc:403] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:19:19.979878 29987 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:19.975013 29996 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:19.970759 29839 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:19.995030 25784 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:19:20.017125 26342 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:20.014200 25778 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:19.984570 29979 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:19.978120 29898 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:20.026283 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 104238us
I20260709 12:19:20.027117 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:19.950647 29993 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 12:19:20.029876 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.030791 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:19.995208 29943 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:20.031590 25785 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:20.020915 25784 pending_rounds.cc:85] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 8
I20260709 12:19:20.033974 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.034930 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.017089 29805 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.021908 29839 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:19.994899 29414 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:19:20.049706 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.042572 25704 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:20.019899 29996 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.035838 25784 pending_rounds.cc:107] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:19:20.056854 25784 raft_consensus.cc:2889] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:20.057250 25784 pending_rounds.cc:107] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:19:20.051694 29984 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.057504 25784 raft_consensus.cc:2889] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:20.030407 29987 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:20.049870 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.036224 29943 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:20.064731 25711 raft_consensus.cc:2410] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:19:20.034225 29921 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:20.060070 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.067960 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.069949 25752 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:20.072873 25704 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:20.073951 25707 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:20.074361 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:20.074719 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 82530.3us
I20260709 12:19:20.056051 29847 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:20.037603 29981 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:19:20.082033 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.069327 29943 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.079422 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:20.083680 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51762.9us
I20260709 12:19:20.084478 29987 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.086411 30007 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:20.086872 29805 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.087755 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.068742 29921 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.091184 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.098748 29874 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.098232 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.075297 29759 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:19:20.109275 29901 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.004s
I20260709 12:19:20.109990 29901 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:20.105388 30007 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.114660 30007 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.116132 29613 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.076457 29998 raft_consensus.cc:2764] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:19:20.118932 25704 raft_consensus.cc:2410] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:19:20.119428 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:20.120376 29888 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:20.120661 29955 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.121150 29888 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:20.121675 29888 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:20.122113 29888 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:20.123682 30010 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:19:20.122211 29999 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:20.079208 29683 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:20.128387 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:20.130802 29979 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.132550 29984 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.107592 29921 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:20.136528 29509 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:20.137017 29509 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.137374 29509 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:20.138163 29966 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:19:20.134271 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46685.9us
I20260709 12:19:20.138656 29966 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:20.139070 29966 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.139101 25752 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.139885 29979 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.140233 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.142055 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.142722 29891 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:20.143236 29891 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.142920 29663 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:20.143637 29891 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.065992 29968 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:19:20.147204 29697 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:20.152985 29697 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:20.147521 29998 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:20.076889 29857 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:20.156263 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48196us
I20260709 12:19:20.164987 25631 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:19:20.167114 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.167791 25634 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:20.171046 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:20.173204 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.175151 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.176596 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.177290 29792 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:20.177609 30007 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.177983 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.168632 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:20.191213 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 111566us
I20260709 12:19:20.180220 30007 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.199355 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.200166 25707 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:20.201756 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.202576 25704 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:20.206976 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.195158 29429 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:20.207787 25706 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:20.178005 29792 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:20.208923 29792 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:20.209329 29792 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.212054 29996 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.213654 26337 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:20.168951 29759 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.215126 29996 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.215366 29759 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.216516 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.217183 25709 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
W20260709 12:19:20.215991 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0996491840001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
W20260709 12:19:20.217653 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0807491840000694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 12:19:20.218014 25676 connection.cc:256] RPC call timeout handler was delayed by -1.0501491840000199s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
I20260709 12:19:20.218143 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.221920 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.232824 25636 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:20.168808 29976 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:20.235530 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.236594 25705 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:20.154004 29697 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:20.237915 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64810.5us
I20260709 12:19:20.239264 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.242059 25712 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:19:20.242600 25712 pending_rounds.cc:85] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 9
I20260709 12:19:20.242871 25712 pending_rounds.cc:107] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:19:20.243152 25712 raft_consensus.cc:2889] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:20.243781 25708 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:19:20.239880 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 190248us
I20260709 12:19:20.244441 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:20.241034 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.223738 29970 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:20.251953 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.232344 29613 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:20.256978 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 75697.9us
W20260709 12:19:20.258277 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 63007.5us
I20260709 12:19:20.240453 29968 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:20.261454 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.245483 29977 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.264654 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.251874 30007 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:20.252657 29970 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.266983 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.270678 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.241024 29913 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 12:19:20.264995 29894 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.650s	user 0.011s	sys 0.000s
I20260709 12:19:20.264276 29979 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:20.278193 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.281453 29509 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.284288 29943 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.284224 29888 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.274430 25636 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.287835 25708 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
W20260709 12:19:20.291857 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51979.6us
I20260709 12:19:20.271703 29962 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:20.291970 25634 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:20.273756 30007 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.295694 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.272863 29955 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:20.273679 29697 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.300053 25677 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.276140 29970 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.305835 29943 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.306614 25630 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.311162 30011 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:20.311718 30011 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.312678 30011 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.302241 30007 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.292511 29979 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.318780 29979 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.319411 29509 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.297363 29962 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.321252 29962 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.297986 29874 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:20.293136 29799 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:20.328372 29799 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.328817 29799 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.329450 29839 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.332862 29888 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.275959 29968 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.334782 29891 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.360194 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.336306 25630 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:20.338785 29987 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:19:20.342729 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.369562 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 121822us
I20260709 12:19:20.367440 25708 raft_consensus.cc:2368] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
I20260709 12:19:20.342177 29921 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.373948 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.374809 25635 raft_consensus.cc:2368] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7.
I20260709 12:19:20.375294 29921 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:20.377451 26308 leader_election.cc:400] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7.
I20260709 12:19:20.377808 26308 leader_election.cc:403] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:19:20.380316 25603 leader_election.cc:400] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
I20260709 12:19:20.380594 25603 leader_election.cc:403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:19:20.357954 29891 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:20.353024 29839 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.358991 29968 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.384960 29921 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
I20260709 12:19:20.379096 29987 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:20.389400 29898 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.410849 29898 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:20.391657 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 132646us
I20260709 12:19:20.396154 25636 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:20.398806 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 338850us
I20260709 12:19:20.381095 29888 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:20.358050 29966 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.402424 29987 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.403522 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:20.421375 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.422356 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.424302 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:20.426242 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:20.427948 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:20.413200 29962 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.442612 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:20.445179 26337 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:20.447100 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.448941 25631 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:20.427342 29898 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:20.430364 29697 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.457850 29697 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.444422 29993 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:20.454672 29891 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:20.462118 29993 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:20.462779 29891 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:20.463472 29891 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.463573 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.464308 26340 raft_consensus.cc:2376] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9. Re-sending same reply.
I20260709 12:19:20.464594 29891 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.465438 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.456453 29898 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:20.466408 29970 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.467348 29898 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.468742 29970 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:20.450651 29894 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.470913 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.466255 26340 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:20.466092 29891 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.474148 29953 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:19:20.475127 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47291.4us
I20260709 12:19:20.475780 29966 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:20.436096 29913 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:20.435892 29839 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:20.462720 29993 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.463045 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:20.486124 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:20.495172 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.495751 25708 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:20.489861 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52930.9us
W20260709 12:19:20.481345 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67178.8us
I20260709 12:19:20.498977 26336 raft_consensus.cc:2368] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8.
I20260709 12:19:20.500296 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:20.430647 29792 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.501641 29913 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:20.501784 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.502667 30018 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.503167 25630 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:20.502787 29913 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.503280 30018 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.500180 26343 raft_consensus.cc:2376] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8. Re-sending same reply.
I20260709 12:19:20.502677 26342 raft_consensus.cc:2368] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:20.505393 30018 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:20.505282 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:20.490706 29894 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.507228 29874 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.507838 29874 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.508347 29894 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.499768 29839 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:20.508984 29839 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.510319 29966 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:20.510931 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:20.512334 29966 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:20.513717 29976 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:20.514138 29976 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.513980 29799 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.515218 29976 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.516129 29792 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:20.508407 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.517073 29976 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.517400 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:20.520614 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.538365 25709 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:20.540318 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.523088 29953 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:20.543208 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.543700 29953 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:20.544044 25634 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:20.544229 29953 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.545379 30022 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:20.546898 29874 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:20.547616 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52683.7us
I20260709 12:19:20.549634 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:20.551334 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.552099 25708 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:20.552414 25629 raft_consensus.cc:2368] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:19:20.550935 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.553325 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.556982 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.560590 29697 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.563849 30022 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.564322 29888 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7.
I20260709 12:19:20.564886 25634 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:20.555743 29979 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.566962 25636 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:20.522368 30007 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.555449 29962 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:20.569880 29964 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:20.570091 29962 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:20.570363 29964 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.570890 29962 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.553857 29792 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:20.571484 29964 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.572031 29792 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:20.572638 29792 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.572939 29964 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:20.573192 30007 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.571980 29962 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.574869 29979 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:20.548568 29968 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.576721 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:20.583460 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:20.585804 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:20.588951 29874 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:20.589529 29874 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.590018 29874 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:20.590812 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.591964 29966 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:20.592878 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.593602 25706 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.593547 29968 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:20.595108 29970 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:20.595486 29970 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.596655 29799 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:20.596335 29970 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.597785 29970 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:20.600173 29953 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.583321 29987 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:20.603628 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:20.604117 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.606829 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.607484 25709 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.607811 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:20.602136 25706 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:20.609834 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.610531 29955 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:20.607059 29993 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.611352 29955 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.558266 29898 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:20.619416 29799 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:19:20.626722 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.620831 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.628811 25709 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:20.625921 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.628453 26337 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.619959 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.630833 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.631596 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:20.634145 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.635452 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:20.635476 25636 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:20.636250 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:20.616792 29898 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.639849 29898 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:20.647712 29966 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:20.650873 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.651695 25634 raft_consensus.cc:2410] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:19:20.653036 29953 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:20.655102 26335 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:20.661260 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.661763 30018 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.648804 29888 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:19:20.662400 30018 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.664454 30018 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.666812 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.667871 29839 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.669055 26341 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:20.628189 29979 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:19:20.670950 29891 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:20.671934 29891 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:20.672279 29891 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.638370 30007 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:20.673108 30007 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.654775 29799 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.674162 29799 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.674261 30007 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.672408 29839 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.675530 29993 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:20.676569 29799 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:20.677776 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.680135 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.682024 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:20.682700 25786 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:20.682824 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.682135 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.652329 29987 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.683820 25782 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:20.685420 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.640187 29921 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:19:20.686545 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53933.1us
I20260709 12:19:20.692409 25780 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:20.693337 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:20.696163 25786 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:20.697086 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.695262 26337 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:20.693933 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.699870 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:20.693953 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:20.698587 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:20.701617 25634 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:20.703548 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.703143 25709 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:20.706107 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:20.707005 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.708199 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:20.709900 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:20.673308 29891 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.712015 25704 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:20.713858 29891 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:20.714599 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.715416 25630 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:20.697623 29966 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:19:20.722003 29966 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:20.722371 29966 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.711570 25783 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:20.722810 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.722015 25707 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:20.725739 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.729246 25752 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.729321 25778 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:20.729945 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:20.730947 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:20.731074 30029 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:19:20.697005 29921 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.732488 25630 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:19:20.733515 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.748451 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.734361 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:20.737242 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46614.6us
I20260709 12:19:20.739813 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.741300 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.743001 25631 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:20.750358 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:20.710381 29799 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:20.745985 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.747892 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:20.751979 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.752089 25785 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:20.751976 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:20.753957 25778 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:20.755568 25637 raft_consensus.cc:2410] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:19:20.757648 25636 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:20.731107 29663 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:20.760207 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.766098 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.767067 29847 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:20.767652 29847 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.727867 30018 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:20.732426 29913 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:20.763998 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.769335 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.770623 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:20.775730 26339 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:20.771548 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:20.776127 26337 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:20.768445 29950 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.778339 26341 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
W20260709 12:19:20.778625 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.759651 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:20.782148 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.780045 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.785640 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.785770 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.786681 25637 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:20.755174 29998 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:20.790519 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.791249 25786 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:20.780342 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.792003 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.792773 25636 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:20.793375 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.764900 29663 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.798025 25631 raft_consensus.cc:2368] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:19:20.811232 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:20.748389 29759 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:19:20.814321 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.814477 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:20.774652 30018 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.734330 29943 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:19:20.819587 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.777526 29799 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:20.839063 25782 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:20.841203 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:20.842258 25637 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:20.839201 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:20.844491 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.790120 29697 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:20.854887 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:20.814072 29663 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.782409 29414 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:19:20.815341 29962 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:19:20.861980 30036 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:19:20.819905 30018 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:20.872901 29921 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.754962 29966 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.874627 30024 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:20.875927 30032 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:19:20.898921 30024 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.860344 25629 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:20.901315 30024 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.879552 29962 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:20.905290 29962 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.906450 25785 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:20.880797 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 96104.5us
I20260709 12:19:20.848096 29874 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.884416 29847 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:20.885411 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78198.8us
I20260709 12:19:20.893827 30011 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:19:20.878266 30027 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:19:20.895118 29966 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.917388 25706 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:20.895380 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 81031.9us
I20260709 12:19:20.895792 29613 raft_consensus.cc:2764] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:19:20.895830 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 63996.4us
W20260709 12:19:20.897936 30022 consensus_queue.cc:1175] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7bf8f20701245bcbaa09cc663025f97" responder_term: 9 status { last_received { term: 9 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 9 index: 7 } }
I20260709 12:19:20.915005 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:20.916154 25780 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:20.878430 29697 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:20.926254 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.923437 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.928392 25781 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:20.933002 25782 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.934810 25786 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:20.940330 25787 raft_consensus.cc:2368] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
I20260709 12:19:20.930325 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:20.942584 25785 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:20.912322 29805 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:20.958874 29799 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.960400 29697 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:20.933806 29792 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:20.968217 29874 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:20.893414 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:20.931569 30011 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:20.981560 25633 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:19:20.970435 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:20.986685 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57743.9us
W20260709 12:19:20.996608 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.990972 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:21.001444 25637 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:20.997243 25782 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:21.005897 30011 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.009495 25784 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:21.013289 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:21.021605 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 145000us
I20260709 12:19:21.023521 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:21.024397 25780 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:21.025799 25785 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
W20260709 12:19:21.026943 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 187797us
I20260709 12:19:21.028396 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:21.031164 25782 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:21.034567 25779 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:20.999694 29805 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:21.042423 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 72061.9us
I20260709 12:19:21.046598 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.027318 29987 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:21.051088 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.040274 29964 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:21.066757 26344 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:21.067489 29976 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:21.077873 25637 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:21.078547 25780 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:21.079725 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:21.050688 29987 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:21.080302 25782 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:21.071934 29984 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:21.092501 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.099864 25782 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:21.081638 29987 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:21.085114 29888 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:21.146550 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:21.216792 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 70409.6us
W20260709 12:19:21.231899 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 185432us
I20260709 12:19:21.285451 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.126596 29993 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:21.289629 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 201504us
I20260709 12:19:21.147939 29888 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:21.219372 29839 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:21.296084 26309 leader_election.cc:336] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.749s (SENT)
I20260709 12:19:21.297107 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.121302 29898 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.299309 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:21.301187 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.298082 30022 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:21.218770 30047 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:21.234412 29874 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:21.296211 29663 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.308305 29663 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:21.308913 29898 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:21.232939 29921 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:19:21.309800 29874 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.311525 26335 raft_consensus.cc:2368] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12.
I20260709 12:19:21.295517 29839 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.309935 29888 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.120891 29805 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:21.319049 30048 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 12:19:21.321591 29950 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:21.297219 30018 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.323807 26339 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:21.324791 30018 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:21.328765 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:21.331809 30060 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.332298 29898 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:21.332321 30060 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.221943 30011 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:19:21.334245 29987 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.330878 29962 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.336024 30060 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:21.336133 30054 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:19:21.337177 29955 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:21.339604 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:21.303184 29792 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:21.358644 25629 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:21.360253 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 55672.8us
I20260709 12:19:21.361317 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:21.364871 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:21.305927 30047 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:21.370793 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:21.376032 25632 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:21.371901 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.374037 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:21.377470 25786 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:21.346256 29950 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:21.388499 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:21.389278 25635 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:21.298308 30046 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:19:21.395452 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:21.337849 29955 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.394994 25781 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:21.386113 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:21.398018 25634 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:21.398085 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.366329 30022 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:21.399385 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.409261 29976 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:21.380347 30047 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.388406 29950 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:21.405534 29792 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:21.411878 29984 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:21.420528 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.433039 29839 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:21.433929 30056 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:21.446705 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:21.447746 25634 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:21.449512 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:21.450012 29847 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:21.450692 29847 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:21.415912 29874 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:21.451854 29847 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.416397 29964 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:21.427000 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:21.455000 25782 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:21.455842 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:21.430884 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:21.456765 25780 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:21.456861 25786 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:21.422816 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:21.459216 25781 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:21.419348 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:21.463958 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50404.4us
I20260709 12:19:21.464900 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.466958 29955 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:21.468570 29962 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:21.462306 25632 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:21.471915 25751 leader_election.cc:400] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8.
I20260709 12:19:21.473121 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:21.431061 29987 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:21.478062 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 182015us
W20260709 12:19:21.479475 26309 leader_election.cc:336] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.783s (SENT)
I20260709 12:19:21.479950 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.426297 29950 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:21.484299 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63914.3us
I20260709 12:19:21.485119 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.486861 25786 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:21.483695 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:21.472677 29874 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:21.491775 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.474771 29955 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:21.482923 29697 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:21.496788 29613 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:21.497133 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:21.499131 29964 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:21.499552 29987 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.500020 29964 consensus_queue.cc:1243] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:21.500191 29987 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.431529 29984 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:21.502701 29984 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:21.502779 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:21.476150 29977 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:21.506057 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:21.504782 30022 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:21.506810 29613 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:21.500674 29964 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:19:21.507169 29966 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 12:19:21.493460 25780 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:19:21.507472 29964 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:21.487784 30045 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.411s	user 0.002s	sys 0.003s
I20260709 12:19:21.508558 29964 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.509346 29955 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:21.474401 30056 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:21.510839 29987 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:21.507494 29613 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.512822 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:21.514140 29613 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:21.514335 29913 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:21.493327 29874 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.516633 29874 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.488953 29962 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:19:21.519826 30061 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.519603 29984 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.520368 30061 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.521760 29984 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:21.514832 29987 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:21.524696 25781 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:21.515672 29964 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:21.514179 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.527281 25635 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:21.528937 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:21.530207 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.531178 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.531869 25705 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:21.532191 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.533193 25710 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:21.533062 25704 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
W20260709 12:19:21.537096 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.539330 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:21.542944 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:21.544435 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.544878 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:21.545475 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 66059.7us
I20260709 12:19:21.547431 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.548796 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.549360 25711 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:21.550246 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.551016 25713 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:21.551160 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:21.546774 26309 leader_election.cc:336] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.794s (SENT)
I20260709 12:19:21.543701 25710 raft_consensus.cc:2376] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:21.551959 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.552470 25704 raft_consensus.cc:2368] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11.
W20260709 12:19:21.553345 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60643.5us
I20260709 12:19:21.553614 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.554378 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.551744 25705 raft_consensus.cc:2376] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:21.548377 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:21.555774 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45836.4us
I20260709 12:19:21.556064 25706 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:21.527135 30056 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:21.526618 29697 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:21.558157 25778 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:21.560608 25601 leader_election.cc:400] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10.
I20260709 12:19:21.536298 29613 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:21.560738 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.561688 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:21.572032 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66093.3us
I20260709 12:19:21.578998 25713 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:21.581377 30007 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:21.543411 29966 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:21.581979 30007 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:21.556654 29888 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:21.583590 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:21.585592 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:21.586278 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.579969 30061 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:21.559005 30046 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:21.589048 25635 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:21.562620 30011 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:19:21.590103 26309 leader_election.cc:336] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.879s (SENT)
I20260709 12:19:21.558657 30063 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.591519 25707 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:21.570391 29839 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:21.553000 29987 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:21.590564 30007 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.608741 29987 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.609812 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:21.590888 30046 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:21.611620 30046 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.615372 30046 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.615970 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:21.595574 29943 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:19:21.616943 25781 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:21.591571 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:21.596764 30024 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:21.606910 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.618526 30011 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:21.618811 25706 raft_consensus.cc:2376] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11. Re-sending same reply.
W20260709 12:19:21.619513 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.011s	sys 0.039s
I20260709 12:19:21.559211 30065 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:21.621044 29857 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:21.621455 29857 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.621800 29857 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:21.623838 25601 leader_election.cc:400] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12.
I20260709 12:19:21.624388 25601 leader_election.cc:403] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:19:21.625582 29921 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:19:21.619946 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.012s	sys 0.039s
I20260709 12:19:21.626879 29921 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12.
W20260709 12:19:21.627471 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 846845us
I20260709 12:19:21.627504 29697 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.618347 30024 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:21.634032 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.635782 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:21.619907 30011 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.639626 29993 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:21.639883 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 49799.8us
I20260709 12:19:21.640235 30065 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:21.640578 26309 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.894s (SENT)
I20260709 12:19:21.641247 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.641762 25778 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:21.647897 25704 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:21.564476 29962 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:19:21.648953 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:21.652786 25712 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:19:21.654304 25780 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:19:21.627110 29759 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:21.580118 29950 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.626672 29894 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:19:21.648656 30060 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:21.658420 29839 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:21.640319 29993 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.660688 29759 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:21.699958 25704 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:21.700994 25708 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:21.645802 30018 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:19:21.562669 30056 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.680882 30060 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.698817 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:21.703745 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 63193.9us
W20260709 12:19:21.706595 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.693920 29799 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.706974 29839 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:21.712335 29759 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.714025 29993 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:21.637485 30063 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.719285 30065 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.735416 29874 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.737677 25635 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:21.742945 30024 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.752287 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:21.763176 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.767302 25713 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:21.811074 26341 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:19:21.751727 30047 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.759704 30068 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:19:21.769168 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:21.820508 26339 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:21.770128 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:21.775223 30067 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:19:21.781150 30056 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:21.782969 29950 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:21.791939 26309 leader_election.cc:336] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.896s (SENT)
I20260709 12:19:21.792528 25710 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:21.800475 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 151380us
I20260709 12:19:21.831419 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:21.807138 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 157237us
I20260709 12:19:21.808149 26344 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:21.817875 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.819324 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:21.833792 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:21.834143 25708 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:21.834685 25710 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:19:21.796680 29663 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.832828 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:21.837335 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:21.844774 25634 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:19:21.845183 25634 pending_rounds.cc:85] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 8
I20260709 12:19:21.845438 25634 pending_rounds.cc:107] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:19:21.845713 25634 raft_consensus.cc:2889] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:21.848095 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:21.808216 29987 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:21.852893 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:21.863207 25635 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:21.819095 29962 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:21.848764 25634 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:21.760847 29799 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.868683 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:21.835961 29759 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:21.761561 30060 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.843232 30046 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.873927 25780 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:21.875332 25784 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:19:21.873320 25787 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:19:21.846642 29979 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.835790 29805 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:21.887058 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:21.893273 25707 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:21.828073 29857 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:21.977895 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:21.983585 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 116860us
I20260709 12:19:21.851261 29663 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:21.986358 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 194463us
W20260709 12:19:21.993115 25535 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 304) took 27957 ms (client timeout 15000 ms). Trace:
W20260709 12:19:21.993238 26309 leader_election.cc:336] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.947s (SENT)
W20260709 12:19:21.955984 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 123225us
I20260709 12:19:21.995347 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:22.018666 30022 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.019622 30022 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:21.950380 29414 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:22.039217 25704 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:22.040278 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.049597 25707 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:21.956632 29898 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.983615 29964 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.874681 29987 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.890937 30046 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.894438 29962 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:22.067332 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 74146.2us
I20260709 12:19:21.880663 30073 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.174s
I20260709 12:19:22.069418 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:21.948043 29979 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.960572 29976 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 12:19:22.100250 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:22.072811 26309 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.997s (SENT) [suppressed 57 similar messages]
I20260709 12:19:22.103262 29857 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:21.949329 29805 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:21.989130 30061 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:19:21.985170 29839 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:22.136787 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64043.3us
I20260709 12:19:22.103968 30070 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:22.057960 29414 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:22.141448 26309 leader_election.cc:336] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.019s (SENT)
W20260709 12:19:22.101943 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 101121us
I20260709 12:19:22.071610 29921 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:19:22.151043 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:22.151716 25707 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:22.153630 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.154803 25712 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:22.157816 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 162566us
W20260709 12:19:21.993744 25535 rpcz_store.cc:269] 0709 12:18:54.035256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.035419 (+   163us) service_pool.cc:224] Handling call
0709 12:19:21.992999 (+27957580us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:19.789715 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:19.790381 (+   666us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:19.790449 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:19:19.790470 (+    21us) tablet.cc:662] Decoding operations
0709 12:19:19.791414 (+   944us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:19.791516 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:19.791542 (+    26us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:19:19.792650 (+  1108us) tablet.cc:701] Row locks acquired
0709 12:19:19.792700 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:19:19.792890 (+   190us) write_op.cc:270] Start()
0709 12:19:19.793044 (+   154us) write_op.cc:276] Timestamp: P: 1783599559792822 usec, L: 0
0709 12:19:19.793089 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 12:19:19.902440 (+109351us) log.cc:844] Serialized 2855 byte log entry
0709 12:19:20.286196 (+383756us) op_driver.cc:464] REPLICATION: finished
0709 12:19:20.522907 (+236711us) write_op.cc:301] APPLY: starting
0709 12:19:20.523042 (+   135us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:20.583121 (+ 60079us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:20.583267 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:20.739661 (+156394us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:20.739970 (+   309us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 12:19:20.740030 (+    60us) write_op.cc:312] APPLY: finished
0709 12:19:20.740387 (+   357us) log.cc:844] Serialized 73 byte log entry
0709 12:19:20.770291 (+ 29904us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:20.784278 (+ 13987us) write_op.cc:454] Released schema lock
0709 12:19:20.806272 (+ 21994us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:20.904550 (+ 98278us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:20.904601 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":485089,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":204184,"apply.run_cpu_time_us":17176,"apply.run_wall_time_us":350993,"cfile_cache_hit":26,"cfile_cache_hit_bytes":27209,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6220,"cfile_init":2,"lbm_read_time_us":3880,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":11,"num_ops":7,"prepare.queue_time_us":79014,"prepare.run_cpu_time_us":17409,"prepare.run_wall_time_us":263066,"raft.queue_time_us":179216,"raft.run_cpu_time_us":17713,"raft.run_wall_time_us":152697,"replication_time_us":492901,"thread_start_us":432294,"threads_started":4,"wal-append.queue_time_us":165110,"wal-append.run_cpu_time_us":26173,"wal-append.run_wall_time_us":1652710}]]}
I20260709 12:19:22.160501 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:22.066654 29964 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.165361 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.137214 29984 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:22.173935 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:22.178787 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:22.179697 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.180490 25709 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:22.184412 30022 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.185339 30090 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:22.186033 30090 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:22.187212 30090 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:22.188957 30078 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:19:22.189965 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.190476 30088 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.191179 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.191875 25713 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:22.193011 25713 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:22.145074 30075 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:19:22.195015 30074 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:19:22.197220 30081 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 12:19:22.163368 25780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:22.198868 25780 raft_consensus.cc:606] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.199733 29913 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:22.200160 29913 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:22.200534 30080 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:19:22.203500 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:22.204535 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.210734 26339 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:22.217777 26344 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:22.164551 29874 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:22.106529 30087 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:19:22.171501 29979 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:22.236986 26309 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.111s (SENT)
I20260709 12:19:22.241139 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.242611 25705 raft_consensus.cc:2410] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:19:22.179339 30063 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:22.206127 30073 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:22.258859 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 98411.4us
I20260709 12:19:22.223096 30070 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:22.253829 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 188317us
I20260709 12:19:22.263136 25713 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:22.249235 25707 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:22.201110 29913 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.265822 29414 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:22.266755 29913 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:22.269747 29964 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.270936 29805 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:22.271656 30071 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.377s	user 0.000s	sys 0.001s
I20260709 12:19:22.262341 30070 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:22.273061 30070 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:22.226300 29894 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 12:19:22.274828 25584 auto_rebalancer.cc:893] Removing replace marker failed: Replica c7bf8f20701245bcbaa09cc663025f97 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:19:22.275435 25584 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:19:22.275776 25584 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:19:22.265717 29663 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.191215 30088 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.279060 29987 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.280470 30088 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:22.281711 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44792.1us
I20260709 12:19:22.245146 29799 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.283838 26339 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:22.285059 26341 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:22.285279 30073 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:22.287145 29962 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:22.175207 29984 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.295903 25710 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:22.289592 25779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:22.297122 26344 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:22.298460 25779 raft_consensus.cc:606] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.298671 25704 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:22.298885 25779 raft_consensus.cc:614] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:22.299705 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.300294 26341 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:22.300403 25706 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:22.303884 26341 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:19:22.304847 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:22.306548 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:22.266009 30046 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.308475 26341 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:22.330834 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:22.332334 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:22.336798 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:22.338052 26335 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:22.344398 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:22.345223 26338 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:22.345379 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:22.345379 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:22.350888 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61835.2us
I20260709 12:19:22.350881 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:22.365355 29898 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:22.366981 30047 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:22.370957 30102 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:22.371299 29970 raft_consensus.cc:993] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:19:22.377923 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:22.378667 29984 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:22.381729 30007 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.003s
I20260709 12:19:22.379607 30096 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.380223 25532 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 4 to 5, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 5 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:22.381000 30070 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:22.381688 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49441us
I20260709 12:19:22.384213 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:22.378633 29987 raft_consensus.cc:2764] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:19:22.387454 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.388913 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.397653 25786 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:22.397832 25782 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:22.399401 25780 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:19:22.403684 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47187.7us
I20260709 12:19:22.405145 25783 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:22.389521 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.391119 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:22.400251 30096 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.408699 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:22.409163 30096 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.409860 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:22.412640 25786 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:22.391448 29964 raft_consensus.cc:1081] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:19:22.391353 29799 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:22.415258 29799 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:22.415674 29799 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:22.419353 30102 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:22.419795 30102 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:22.420647 30102 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.422207 30102 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:22.395424 30007 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.424144 25676 leader_election.cc:400] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:22.424521 26339 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:22.424747 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:22.425176 30061 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:22.425570 30092 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:22.396143 30018 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:22.446787 30007 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:22.431022 29874 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:22.431815 30074 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:22.448832 30074 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:22.449261 30074 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:22.450595 30047 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:22.433203 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.452550 30070 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.452875 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.453531 29898 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:22.453431 30083 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:22.454052 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.442732 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.441497 25676 leader_election.cc:403] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:19:22.457713 30096 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:22.457878 30063 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:22.458382 30063 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:22.474794 26344 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:19:22.476346 29979 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:22.466715 29987 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:22.464133 30018 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:22.467299 30047 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:22.486323 29847 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:22.486518 25532 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 4 to 5, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 5 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:22.469830 30007 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 12:19:22.493391 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 69513.2us
I20260709 12:19:22.459306 30063 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.494067 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:22.459712 29663 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:19:22.491226 25532 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:22.469481 30092 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:19:22.507939 29987 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:22.502442 29943 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:22.508985 29943 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:22.510058 30063 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:22.510394 26344 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:22.450191 30074 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.497251 30047 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:22.518230 29970 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:22.519018 30047 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.509953 29943 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.521652 29943 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:22.523690 25676 leader_election.cc:400] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8.
I20260709 12:19:22.524081 25676 leader_election.cc:403] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:22.527772 30074 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:22.531405 29987 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8.
I20260709 12:19:22.531236 29799 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.535468 29799 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.541229 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:22.541988 26336 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:22.550431 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.551254 30007 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:22.554067 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.560595 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.562426 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.554811 25712 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:22.568848 25712 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:22.569630 25706 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:19:22.573813 25713 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:19:22.577657 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54111.7us
I20260709 12:19:22.578781 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:22.569445 29792 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.583734 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:22.584780 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.586437 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:22.587728 25629 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:22.583765 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:22.572640 29964 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.566056 29979 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:22.591114 29792 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.573634 29970 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.597668 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.590423 30070 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.595362 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.596724 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.662021 25785 raft_consensus.cc:2376] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10. Re-sending same reply.
W20260709 12:19:22.615118 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.597273 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.663012 25786 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:22.626828 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:22.595187 25633 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:22.590413 29613 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:22.655040 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.666764 25781 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:22.672482 30088 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.673113 30088 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.674762 29792 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.593084 29976 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:22.675261 30088 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.675581 29976 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:22.676102 29976 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:22.662163 25784 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:22.678922 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.680125 25784 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:22.681010 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.682147 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.676100 30088 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:22.682699 30069 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:22.590045 29759 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:22.695268 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 111574us
I20260709 12:19:22.695369 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:22.683236 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.696368 26344 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:22.685928 25784 raft_consensus.cc:2410] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:22.687788 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:22.697698 26336 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:22.688346 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.688472 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:22.699671 25786 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:22.697978 25638 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:22.689577 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.699980 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:22.701262 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:22.701822 25781 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:22.702752 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:22.597056 29984 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:19:22.703688 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.640268 30097 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:22.698975 30106 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:19:22.708750 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:22.710192 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.695575 29950 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:22.720566 25781 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:22.727401 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.728417 25784 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:22.654302 29979 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.641093 29964 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.699848 30088 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.683815 29970 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.751930 25786 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:22.689452 30070 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.726462 29613 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:19:22.717239 29962 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:22.698038 29857 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:19:22.700845 29759 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.772012 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.774330 25784 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:22.747088 29987 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:22.725404 30092 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:19:22.720455 29993 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:19:22.791324 25781 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:22.771534 29613 consensus_queue.cc:1243] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:22.794580 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.771763 29962 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:22.795913 25784 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:19:22.753985 29894 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:19:22.721974 30097 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.809302 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:22.796444 29966 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.823634 25787 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:22.825863 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.828154 25781 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:22.828387 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 119740us
I20260709 12:19:22.829324 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:22.830554 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:22.806627 30098 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:19:22.800011 30022 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.843364 29979 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.843842 30022 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.844511 29759 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.844823 29970 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.846323 29993 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:22.847992 25787 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:22.784855 29976 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:22.849009 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:22.849713 25787 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:22.850035 29964 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.801041 29962 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:22.831138 25637 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:22.850476 29976 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.852154 30070 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.852586 25785 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:22.832888 30087 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:22.853165 30087 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:22.853569 30087 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:22.856648 30111 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:19:22.858691 30088 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:22.859205 25637 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:22.863600 29987 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:22.831852 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:22.835654 29901 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:22.864352 30097 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:22.830848 29943 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:19:22.833261 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:22.866299 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.868949 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:22.869721 26338 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:22.871006 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.871996 25750 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:22.875166 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:22.880926 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.881721 25708 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:22.883841 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.884142 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:22.884811 25705 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:22.885874 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:22.891119 25603 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:22.876377 29977 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:22.834399 29966 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:22.881870 30070 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:22.884660 29901 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:22.884840 29759 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:22.887970 30022 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:22.890873 30047 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:19:22.931217 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.969863 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51202.3us
I20260709 12:19:23.037087 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:23.039889 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.040879 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:23.041472 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.041596 25630 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:23.041474 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:23.042793 25638 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:22.916968 29977 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:22.916689 29943 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:22.925544 29979 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:23.053340 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:22.969784 29792 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.056663 29792 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.059451 29792 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:23.068750 29966 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.050997 29977 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.043618 29759 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:23.072225 29759 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:23.076107 30118 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:19:23.078510 30049 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.079183 29979 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:23.079407 29943 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:23.037110 30103 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:19:23.080376 29943 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:23.083635 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46660.4us
I20260709 12:19:23.054919 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.085592 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:23.087219 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:23.037575 30047 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.092069 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.093427 25630 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:19:23.098315 26338 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:23.103626 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:23.105894 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2448) took 1245 ms. Trace:
I20260709 12:19:23.079706 29979 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:23.107059 25781 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:23.106320 25778 rpcz_store.cc:276] 0709 12:19:21.853034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.866172 (+ 13138us) service_pool.cc:224] Handling call
0709 12:19:21.866571 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.984918 (+118347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.984964 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.995618 (+ 10654us) log.cc:844] Serialized 26 byte log entry
0709 12:19:22.145931 (+150313us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:22.145988 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.093388 (+947400us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b5400099554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:23.093455 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.093520 (+    65us) raft_consensus.cc:1685] finished
0709 12:19:23.093555 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.098045 (+  4490us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41462,"raft.queue_time_us":32607,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12326,"spinlock_wait_cycles":68844800,"thread_start_us":74147,"threads_started":1,"wal-append.queue_time_us":197142}
I20260709 12:19:23.048694 29894 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:19:23.099787 30047 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:23.079103 30049 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.113885 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:23.138823 25778 raft_consensus.cc:2368] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13.
I20260709 12:19:23.108227 29984 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:23.142562 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:23.140389 30074 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:23.143250 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:23.144529 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57416.4us
I20260709 12:19:23.164685 30120 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:23.143960 29964 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.165860 29964 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.166910 30060 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:23.168179 29964 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.155920 30074 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.172878 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:23.165380 30121 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 12:19:23.191565 30102 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.001s
I20260709 12:19:23.202054 30120 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:23.202816 30120 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.203775 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:23.204674 25781 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:23.205781 30120 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:23.215997 29962 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.227483 30049 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:23.229475 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:23.231858 25781 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:19:23.202142 30074 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:23.240696 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:23.242010 30128 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:23.242612 30128 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.243289 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:23.244310 29613 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:23.244757 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:23.244915 30128 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.245546 29984 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:23.245716 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:23.245054 29613 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:23.246577 29984 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:23.251585 30074 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.252415 30074 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.256199 30074 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:23.279116 25638 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:23.285609 29964 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.298442 25781 pending_rounds.cc:85] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 5
I20260709 12:19:23.299175 25781 pending_rounds.cc:107] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:19:23.306692 25781 raft_consensus.cc:2889] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:23.306738 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:23.312646 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 67008us
I20260709 12:19:23.316586 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:23.331323 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 673) took 1163 ms. Trace:
I20260709 12:19:23.289121 29962 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.332062 25704 rpcz_store.cc:276] 0709 12:19:22.152395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.152754 (+   359us) service_pool.cc:224] Handling call
0709 12:19:22.298319 (+145565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.299032 (+   713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.315808 (+1016776us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400105750. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:23.315975 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333936000}
I20260709 12:19:23.336366 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.337076 25712 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:23.337924 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.333109 30120 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:23.340925 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.341503 25707 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:23.341763 25781 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:19:23.343498 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.343870 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.289811 29977 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.299152 30087 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.323226 29979 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.347188 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.354421 25704 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:23.339417 30047 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.356170 30128 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:23.356884 30047 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.352198 30087 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:23.360735 25707 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:23.361593 29979 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.356849 30128 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.363796 25704 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:23.340008 29613 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:19:23.364569 30128 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:23.367385 25633 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:23.309302 29964 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.372140 25638 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:23.373394 25630 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:23.334025 29984 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:19:23.375523 25638 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:23.375711 25781 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:19:23.376704 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:23.377210 26338 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.377987 25635 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:19:23.366937 29977 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.381938 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.382671 25638 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:23.383831 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.384378 25635 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:23.383419 29979 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.387313 29979 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.392184 29964 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:23.392549 29979 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.399451 29913 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:23.404990 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:23.410920 25782 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:23.406751 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:23.412956 26338 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:23.416057 29984 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:23.416247 30128 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:23.410334 29913 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:23.419198 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:23.419508 29913 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.425021 25778 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:23.418097 25750 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:23.426286 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.426879 25704 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:23.432765 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.433334 25707 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:23.433820 25782 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:23.434442 25778 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:23.434945 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:23.435437 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:23.436147 25782 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:23.436822 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:23.440047 29977 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:19:23.440718 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.419981 25783 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:23.442276 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.443363 25705 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:23.444780 25713 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:23.445225 25708 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:23.445868 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.447260 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.448601 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.452833 25704 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:23.454247 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.454447 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.442953 25712 raft_consensus.cc:2376] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8. Re-sending same reply.
I20260709 12:19:23.456269 25707 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:23.456449 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.454895 25708 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:23.457336 25635 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:23.459394 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:23.460506 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:23.461565 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:23.462913 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.465173 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.437043 30024 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.470919 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.471639 25707 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:23.466634 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:23.466594 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.472594 25637 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.467737 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:23.425791 25711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:19:23.474251 25711 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:23.474766 25711 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:23.455906 29913 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.476023 26336 pending_rounds.cc:85] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 7
I20260709 12:19:23.480968 26336 pending_rounds.cc:107] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:19:23.481453 26336 raft_consensus.cc:2889] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:23.472723 25705 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:23.481679 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:23.463621 29888 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:23.483412 29888 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.485598 29888 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:23.486159 29888 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:23.479105 30120 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:19:23.489579 29697 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:23.490007 29697 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:23.477010 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:23.493146 26338 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:23.478209 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.458194 29966 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:23.457829 29977 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:23.498245 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.496986 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.497802 29966 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.505041 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:23.471175 30100 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:23.513340 25781 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:19:23.513643 25638 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:23.535496 25781 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:23.493654 29888 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:23.544639 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63088.5us
I20260709 12:19:23.545507 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.550525 25713 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:23.546676 25638 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:23.471890 30086 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:19:23.470314 30074 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:19:23.498751 30060 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:23.503738 29966 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:23.513218 30100 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.561713 25637 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:23.535871 30022 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:23.547097 29888 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:23.565063 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 103592us
I20260709 12:19:23.567188 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.572182 30093 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:23.572602 30093 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:23.573421 30093 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.579876 25711 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.589092 25713 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:23.591334 25711 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.563660 30100 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.596082 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.596745 25705 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:23.597781 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46357.2us
I20260709 12:19:23.605582 25637 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:19:23.587323 30093 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:23.566401 30060 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:23.612383 30140 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:23.615595 30139 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:23.537092 29697 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.608392 29792 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.582268 30022 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.652956 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.654032 25637 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:23.591190 29977 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.665618 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.666635 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.673141 25637 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:23.673808 25630 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:19:23.674131 25630 pending_rounds.cc:85] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 10
I20260709 12:19:23.674343 25630 pending_rounds.cc:107] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:19:23.674625 25630 raft_consensus.cc:2889] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:23.665189 30137 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 12:19:23.681380 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 112626us
I20260709 12:19:23.685082 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:23.694672 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 95296.7us
I20260709 12:19:23.652287 29792 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.678153 29613 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:23.676802 30086 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:19:23.677037 30083 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 12:19:23.746948 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:23.754554 30141 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:19:23.732427 30135 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:19:23.770418 30074 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:23.800376 30074 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:23.722080 29613 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.804307 29979 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.804904 29979 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.805322 29613 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:23.806449 25713 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:23.807024 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.807126 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.807668 25707 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:23.807796 25711 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:23.808264 25713 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:23.808472 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.809329 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:23.816520 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:23.817205 25634 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:23.708781 30060 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.788125 29792 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.689414 30146 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:19:23.788290 29799 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.788830 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:23.825336 29979 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:23.769361 30149 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:23.792896 30022 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:23.793494 30148 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.062s	user 0.003s	sys 0.000s
I20260709 12:19:23.794929 25705 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:23.795238 29966 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.827849 25603 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:23.789392 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2640) took 1230 ms. Trace:
I20260709 12:19:23.858072 25710 rpcz_store.cc:276] 0709 12:19:22.559145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.559268 (+   123us) service_pool.cc:224] Handling call
0709 12:19:22.559584 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.566070 (+  6486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.566117 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.566731 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.403286 (+836555us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b18003daf80. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:19:23.462412 (+ 59126us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:23.462458 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.462568 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.732125 (+269557us) raft_consensus.cc:1685] finished
0709 12:19:23.732159 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.789303 (+ 57144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2109,"raft.queue_time_us":123,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4099,"spinlock_wait_cycles":3196672,"thread_start_us":58924,"threads_started":1,"wal-append.queue_time_us":969868}
W20260709 12:19:23.832526 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 110186us
I20260709 12:19:23.860723 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:23.836717 25711 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
W20260709 12:19:23.874766 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.876184 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:23.880455 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:23.881632 25710 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:23.891251 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1930) took 1212 ms. Trace:
I20260709 12:19:23.871979 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:23.836405 29792 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:23.836592 29799 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:23.832314 29857 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:19:23.901778 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:23.836095 30086 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:19:23.910456 25603 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:23.894671 30087 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:23.797211 30100 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.900413 25784 rpcz_store.cc:276] 0709 12:19:22.678400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.806899 (+128499us) service_pool.cc:224] Handling call
0709 12:19:22.807284 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.807519 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:22.832129 (+ 24610us) log.cc:844] Serialized 14 byte log entry
0709 12:19:23.492610 (+660481us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b18003e1e80. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a 00007f29b5999a2a
0709 12:19:23.890738 (+398128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:23.890799 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:23.890861 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.890964 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.891162 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22026,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":45,"raft.queue_time_us":69348,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":8409,"spinlock_wait_cycles":16648064,"thread_start_us":380070,"threads_started":2,"wal-append.queue_time_us":809502}
I20260709 12:19:23.897819 29977 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:23.916581 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:23.894954 30007 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:19:23.921706 30087 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.934906 30087 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.938633 29663 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.929378 29805 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:23.939859 29805 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.939424 29663 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:23.940290 29805 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:23.932039 29966 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:23.910107 30151 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:23.913563 29792 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:23.930063 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:23.942939 25637 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.912101 30011 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:23.960229 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:23.963287 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 278463us
I20260709 12:19:23.922236 29970 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:23.964737 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 54390.6us
I20260709 12:19:23.965377 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:23.988322 25637 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:24.012754 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:23.975193 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:23.983490 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:23.967183 30056 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:24.015463 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55298.2us
I20260709 12:19:24.031798 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.033241 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.034072 25601 leader_election.cc:400] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13.
I20260709 12:19:24.032461 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.038532 25601 leader_election.cc:403] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:24.032719 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 337099us
I20260709 12:19:23.964020 29977 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:24.055593 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:24.046226 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.044183 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.044858 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:24.062844 25712 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:24.060662 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.007988 29799 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:24.061064 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.062961 25704 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:24.063629 25631 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:23.964320 29888 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.029349 30150 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:24.014850 30083 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:24.066735 25633 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:23.964187 30011 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:24.051100 29970 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:24.029383 30149 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:24.051158 30151 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:24.078368 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.080734 25704 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:24.039398 30164 raft_consensus.cc:2764] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:19:24.057040 30047 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13.
I20260709 12:19:24.048174 30056 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.046404 30022 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:24.082772 29913 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.004s
I20260709 12:19:24.092839 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.073326 29799 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:24.101222 25707 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:24.076530 30156 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:24.087541 30151 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.008441 29792 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.097540 29966 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.116575 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.093858 29970 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:24.118165 29891 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.091267 30083 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:19:24.102726 30158 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:24.045682 30153 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:19:24.090996 30149 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.069197 30163 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:19:24.098109 30022 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.112149 30151 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:24.098372 30056 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.135347 30176 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:24.159111 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:24.137944 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 106249us
I20260709 12:19:24.160923 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.163987 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:24.126397 30156 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.106945 29799 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.136323 29663 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.174659 29891 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:24.176036 30142 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.487s	user 0.002s	sys 0.000s
I20260709 12:19:24.176663 30174 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:24.171370 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43806.7us
I20260709 12:19:24.188711 30145 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:24.235460 30145 consensus_queue.cc:1243] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:24.190368 30060 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:24.236675 30145 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.193348 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.195243 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.197161 26339 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:24.199810 29950 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:24.239112 29950 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:24.239507 29950 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:24.239809 30098 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:24.240877 30088 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:24.240685 29950 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.241230 30088 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:24.241878 29950 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:24.200994 29913 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:24.242178 30088 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.243341 30088 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:24.201957 30171 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:24.244074 30171 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:24.244439 30171 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:24.245177 30171 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.202863 30155 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:24.246819 30155 consensus_queue.cc:1243] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:24.203678 29901 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:24.206377 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.207036 30163 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:24.207999 30181 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:24.249626 30181 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.250005 30181 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:24.212306 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.213150 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.217056 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:24.205597 30152 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.412s	user 0.001s	sys 0.000s
I20260709 12:19:24.220924 29987 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.198284 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:24.226830 29663 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.127542 29966 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.237125 30145 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.237910 26336 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:24.239292 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.246273 30171 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:24.295034 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 96832.9us
I20260709 12:19:24.308418 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:24.224921 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:24.292402 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:24.301230 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2453) took 1683 ms (client timeout 1569 ms). Trace:
I20260709 12:19:24.168254 30164 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:24.306437 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:24.313143 25632 raft_consensus.cc:2410] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:19:24.176195 30149 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.180567 30150 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:24.311504 25785 rpcz_store.cc:269] 0709 12:19:22.608460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.682995 (+ 74535us) service_pool.cc:224] Handling call
0709 12:19:24.291896 (+1608901us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54002b4654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:24.292061 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357669760}
I20260709 12:19:24.315042 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.230232 30176 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.131711 30011 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.234525 30174 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.139389 30158 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.353472 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:24.299896 29913 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.394228 26336 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:24.394141 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:24.232462 29891 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.146844 29977 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.398650 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:24.397150 25784 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:24.400581 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:24.397475 30024 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.912s	user 0.012s	sys 0.003s
I20260709 12:19:24.177655 30156 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:24.419795 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.397081 30155 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:24.392674 30174 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:24.432766 26336 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:24.379873 30149 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.452878 30149 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.455060 30149 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:24.434252 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 203795us
I20260709 12:19:24.301817 29663 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.320281 29901 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:24.399178 29888 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:24.442036 30127 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:19:24.443121 30167 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
W20260709 12:19:24.444334 30165 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
W20260709 12:19:24.444415 30172 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 12:19:24.445175 30170 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
W20260709 12:19:24.445619 30161 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 12:19:24.445833 30173 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.446201 29993 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:24.447435 30120 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:24.351141 30176 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.449528 30056 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:24.451650 30145 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:24.505851 30169 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 12:19:24.506270 30056 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:24.506875 30056 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.509049 30056 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:24.452458 30162 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 12:19:24.399497 30007 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:24.491230 30127 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.511504 30127 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:24.300180 29987 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.457834 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.329058 29962 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:24.519608 29962 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.450332 29697 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:24.439805 30075 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.494793 26342 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:24.320514 29970 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.329656 30164 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.495877 30150 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.488445 29901 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:24.505511 30188 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.158s	user 0.001s	sys 0.001s
I20260709 12:19:24.518090 29987 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.536079 29663 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.486990 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:24.524255 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.511518 30007 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:24.530362 30087 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.002s
I20260709 12:19:24.530632 29966 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:24.504194 30173 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.560776 30173 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:24.522282 30181 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:24.567600 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.500250 29993 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:24.533340 30075 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:24.491791 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 266963us
I20260709 12:19:24.585804 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:24.519080 29805 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.002s
I20260709 12:19:24.553064 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:24.548049 30158 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:24.558625 30087 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.555646 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.604398 25630 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:24.604595 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:24.605000 26338 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:24.562311 30007 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:24.567044 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.574054 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.545168 30022 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:24.548473 29901 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:24.586967 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.588899 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:24.569824 30189 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:19:24.572504 30190 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:24.558652 30164 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.435158 29913 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.605829 25635 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:24.615089 25784 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:24.616454 25709 pending_rounds.cc:85] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 7
I20260709 12:19:24.652307 25709 pending_rounds.cc:107] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:19:24.652690 25709 raft_consensus.cc:2889] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:24.616715 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:24.621372 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.005s	sys 0.013s
W20260709 12:19:24.648897 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 321109us
I20260709 12:19:24.604031 29962 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:24.654565 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.008s	sys 0.013s
W20260709 12:19:24.656534 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 563770us
I20260709 12:19:24.609014 30171 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:24.655333 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:24.666391 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:24.635228 30061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.000s	sys 0.031s
I20260709 12:19:24.659125 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:24.673301 25785 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:24.673743 25785 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:24.673817 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:24.676018 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 275594us
I20260709 12:19:24.677973 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:24.634938 30190 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.680927 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.636053 30176 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:24.648016 30193 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:24.683820 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:24.684537 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.603616 29805 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:24.672511 30162 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:19:24.691852 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.686620 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.685571 25711 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:24.680230 25785 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:24.675371 30061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.000s	sys 0.032s
I20260709 12:19:24.690358 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.754921 25706 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:24.690647 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.686378 30190 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:24.692934 30149 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:24.689795 30182 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:24.718336 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:24.761473 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 155012us
I20260709 12:19:24.765139 25712 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:24.720099 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.680102 29993 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.721256 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:24.688985 25704 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:24.723541 30087 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.687474 30022 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:24.725996 30181 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:24.726563 30156 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.688738 30193 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.672488 30158 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.722307 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.731132 25638 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:19:24.747467 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.748557 26337 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:24.753620 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.753790 30198 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:24.759342 25785 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:24.749240 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.797403 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.798024 25636 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:24.798852 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.799826 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.760368 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.753518 30197 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.778112 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:24.767210 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.801055 25631 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:24.779096 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.786751 25704 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:24.728042 29805 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:24.794034 25629 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:24.803442 25634 raft_consensus.cc:2410] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:19:24.801710 25630 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:24.754556 30173 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:24.763967 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 108726us
I20260709 12:19:24.809149 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.810817 25636 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:24.812211 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.812846 25629 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:24.817894 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.814186 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.824519 25785 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:24.791843 30176 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.832213 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:24.827525 30022 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:24.792074 30193 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:24.805577 30198 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:24.848505 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.832068 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.840936 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.807757 30181 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:24.812263 30156 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:24.843608 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:24.846099 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:24.812947 29993 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:24.860404 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:24.834545 30127 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.834589 29966 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:19:24.876120 30022 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:24.878221 30147 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.091s	user 0.008s	sys 0.002s
I20260709 12:19:24.880995 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:24.804760 30158 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.871448 30097 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.816164 30173 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:24.896313 30203 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:19:24.897208 30192 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:24.898175 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:24.903344 25631 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:24.835052 29913 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:24.894662 30187 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
W20260709 12:19:24.977667 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 190991us
W20260709 12:19:24.930423 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49508.1us
I20260709 12:19:24.872071 30198 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:24.871380 30140 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.975797 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.981717 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:24.842933 30176 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:24.812438 30197 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:24.988875 30191 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:19:24.989259 30206 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.132s	user 0.001s	sys 0.001s
I20260709 12:19:24.992648 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:24.968183 30153 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:24.977953 30011 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:24.988266 30174 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.015s
I20260709 12:19:24.999783 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:25.008929 25785 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:25.009474 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2469) took 1967 ms. Trace:
I20260709 12:19:25.010459 25782 rpcz_store.cc:276] 0709 12:19:23.042098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.437714 (+395616us) service_pool.cc:224] Handling call
0709 12:19:23.437971 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.441042 (+  3071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.441075 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.470483 (+ 29408us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.640732 (+170249us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:23.640772 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.645191 (+  4419us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.877403 (+1232212us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:25.009188 (+131785us) raft_consensus.cc:1685] finished
0709 12:19:25.009209 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.009412 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5403,"raft.queue_time_us":2768,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":5619,"spinlock_wait_cycles":13474176,"thread_start_us":127135,"threads_started":1,"wal-append.queue_time_us":231104}
I20260709 12:19:24.970817 30097 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.019112 29977 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:25.002694 30056 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:25.000870 30061 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:24.995798 30088 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:19:25.013417 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:25.049304 26340 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:25.010712 30196 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:25.020660 30151 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.787s	user 0.006s	sys 0.006s
I20260709 12:19:25.048578 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:25.069254 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 55904.4us
I20260709 12:19:24.987612 30192 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.075942 26335 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:24.999673 30140 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.083279 30185 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.040640 30156 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:25.079483 30179 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:25.071449 30196 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.071828 29966 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:25.028848 29977 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.068894 30061 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:25.087829 30127 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.028450 30210 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:25.011226 30174 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.075237 29913 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:25.011967 30011 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.090076 30179 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:25.087690 30185 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.143961 25779 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:25.115594 29977 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:25.136888 30210 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.146016 30185 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.118472 30216 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.146771 30185 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.146840 30216 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.147678 30185 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.149396 30216 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.150403 30185 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.135139 29913 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.152521 30210 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.152870 29913 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.154649 30174 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.155864 30173 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.155720 30216 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.156523 30216 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.157874 30074 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:25.158334 30074 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:25.159346 30216 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.159981 30216 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.159507 30074 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.160462 30216 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.160900 30074 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:25.162606 30215 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:19:25.163149 30176 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.164415 30061 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.166618 29913 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.167171 29913 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:25.137977 30207 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:19:25.170156 30216 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:25.171317 30202 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:19:25.093012 30156 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.094004 30196 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:25.177189 30158 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.167150 30197 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.183466 25631 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:25.185034 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:25.188869 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:25.075870 30151 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.143262 30179 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:25.195940 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:25.221654 30214 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:19:25.224107 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:25.196121 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.195006 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:25.232415 25780 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:25.233467 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:25.242015 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:25.246985 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.249498 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.177222 30192 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.253827 25786 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:25.247570 25704 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:25.197111 30184 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.195101 30127 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.217003 30210 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:25.261618 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:25.236192 29901 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.006s
I20260709 12:19:25.216185 30216 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:25.271436 25704 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:25.276041 29891 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.235478 29805 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:25.291548 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.292333 25710 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:25.273725 29697 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.735s	user 0.013s	sys 0.007s
W20260709 12:19:25.299434 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 221079us
I20260709 12:19:25.308064 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.280496 30097 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.311987 25704 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:19:25.321856 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.326396 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.271358 30127 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.334061 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:25.352630 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 171844us
I20260709 12:19:25.280637 29979 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:25.309947 30164 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.356827 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.365118 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.374663 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.375998 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.377197 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.377908 25629 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:25.273808 29901 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.371548 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.294624 30216 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.379217 25713 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:25.379240 25631 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:25.380467 29901 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.380846 30216 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.381142 29901 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.381039 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:25.381791 29901 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.384095 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.324093 30022 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.316381 30210 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:25.353078 29970 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.271585 30184 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.355142 30192 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.355449 30075 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.390275 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:25.415817 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.504005 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.485816 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 288381us
I20260709 12:19:25.556159 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:25.557946 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 68625.4us
I20260709 12:19:25.560562 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:25.563261 25704 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:25.564672 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.565430 25704 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 5.
W20260709 12:19:25.486538 29993 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.003s
I20260709 12:19:25.410989 30140 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.567545 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:25.596959 26335 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:25.604583 25629 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:25.604470 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.490396 30011 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.552724 30088 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:25.422811 29970 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.411456 30164 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.619246 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:25.615445 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:25.596196 30229 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:25.460582 30192 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.597836 30166 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:25.631142 30088 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.584527 30226 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.784301 30192 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:25.784873 30088 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.784786 30226 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.689954 25629 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:25.710528 29993 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:25.715371 30230 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 12:19:25.716356 30222 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
W20260709 12:19:25.727109 30201 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.009s	user 0.000s	sys 0.005s
W20260709 12:19:25.727347 29792 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:19:25.706845 30128 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.729020 26340 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:25.829583 30128 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:25.730242 26338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 760) took 1534 ms (client timeout 1636 ms). Trace:
I20260709 12:19:25.732380 30155 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:25.830765 30155 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:25.830657 26338 rpcz_store.cc:269] 0709 12:19:24.194733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.194961 (+   228us) service_pool.cc:224] Handling call
0709 12:19:25.728449 (+1533488us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b540065c2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:25.728607 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843592448}
I20260709 12:19:25.831673 30155 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.832337 30128 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.832646 30155 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:25.833649 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:25.419203 30022 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.723196 29913 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.621660 30210 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.567194 30228 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:25.792245 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 176880us
I20260709 12:19:25.795321 30171 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.790593 30229 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.764564 30216 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:25.800920 25629 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
W20260709 12:19:25.801780 30236 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:19:25.805270 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.873009 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:25.873562 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:25.805984 30193 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.923s	user 0.011s	sys 0.000s
I20260709 12:19:25.809316 30192 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:25.825451 29993 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.762353 29950 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.815145 29901 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.896853 25629 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:25.820101 30184 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.820561 29970 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.821580 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.907109 30182 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.907224 25634 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:25.907862 30146 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:25.907701 30182 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.908406 30146 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.908596 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.909608 25634 raft_consensus.cc:2393] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:25.821834 30151 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.773774 30156 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:25.824820 30074 consensus_meta.cc:287] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.663s	user 0.015s	sys 0.006s
I20260709 12:19:25.791301 30176 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:25.865351 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:25.833098 30128 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.892967 30164 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:25.861887 29977 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.717s	user 0.006s	sys 0.006s
I20260709 12:19:25.898067 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.934406 25629 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:25.934267 30176 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.935699 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.936765 30176 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.936825 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.937749 25629 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:25.900893 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.893967 30216 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.901458 29950 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.956622 29950 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.893921 30192 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.959280 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:25.959698 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2011) took 1150 ms. Trace:
I20260709 12:19:25.905366 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1067) took 1727 ms. Trace:
I20260709 12:19:25.960556 26344 rpcz_store.cc:276] 0709 12:19:24.178204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.178379 (+   175us) service_pool.cc:224] Handling call
0709 12:19:24.186946 (+  8567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.553093 (+366147us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:24.597906 (+ 44813us) log.cc:844] Serialized 14 byte log entry
0709 12:19:24.689344 (+ 91438us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:24.689393 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:24.689434 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.905104 (+1215670us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b540055a354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:25.905148 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.905300 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45786,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":9057,"raft.queue_time_us":1943,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":23489,"spinlock_wait_cycles":841305344,"thread_start_us":89755,"threads_started":1,"wal-append.queue_time_us":155642}
I20260709 12:19:25.861446 29913 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:25.959276 30192 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:25.962598 29913 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.964705 29913 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.964771 30229 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:25.897799 30196 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.000s
W20260709 12:19:25.930538 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 65284.5us
I20260709 12:19:25.949937 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.897486 30193 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.953023 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.873005 30171 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.973492 30193 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.973664 30171 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.974175 25712 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:25.954896 30216 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.951105 30128 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.975425 30022 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:25.975840 30182 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.961609 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:25.953990 30190 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.167s	user 0.012s	sys 0.000s
I20260709 12:19:25.939241 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:25.933388 26336 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:25.978946 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.979907 25704 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:25.896077 25637 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:25.965436 30196 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:25.970989 30176 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:25.971653 29993 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:25.971910 30179 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.000s
I20260709 12:19:25.953625 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 751) took 1560 ms. Trace:
I20260709 12:19:25.960112 25630 rpcz_store.cc:276] 0709 12:19:24.809214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.809350 (+   136us) service_pool.cc:224] Handling call
0709 12:19:24.827778 (+ 18428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.973957 (+146179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.974005 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.993363 (+ 19358us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.142275 (+148912us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:25.142314 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.142379 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.958311 (+815932us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b2400121b08. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27289c4 00007f29b2726acb 00007f29b28e1dc9 00007f29b28e1a55 00007f29b28e0fb0 00007f29b5557488 00007f29b5557be8 00007f29b552f398 00007f29b552a76a 00007f29b5529bb6
0709 12:19:25.959423 (+  1112us) raft_consensus.cc:1685] finished
0709 12:19:25.959442 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.959639 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148740,"raft.queue_time_us":137502,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":8800,"spinlock_wait_cycles":13931904,"thread_start_us":117666,"threads_started":1,"wal-append.queue_time_us":217485}
W20260709 12:19:25.989282 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.987746 25634 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:25.991621 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:25.960409 30198 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.961s	user 0.010s	sys 0.000s
I20260709 12:19:25.988171 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:25.873652 30228 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:25.994837 30225 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:25.952235 29970 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:25.988894 25787 rpcz_store.cc:276] 0709 12:19:24.392722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.392881 (+   159us) service_pool.cc:224] Handling call
0709 12:19:24.421066 (+ 28185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.421695 (+   629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.421730 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.455028 (+ 33298us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.602986 (+147958us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:24.603042 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.603131 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.725391 (+1122260us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b2400243d58. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27289c4 00007f29b2726acb 00007f29b28e1dc9 00007f29b28e1a55 00007f29b28e109d 00007f29b551310e 00007f29b552a88c 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221
0709 12:19:25.938747 (+213356us) raft_consensus.cc:1685] finished
0709 12:19:25.938768 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.953546 (+ 14778us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72587,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":5836,"spinlock_wait_cycles":53463808,"thread_start_us":147548,"threads_started":1,"wal-append.queue_time_us":234581}
W20260709 12:19:26.003242 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.008630 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:25.995477 30022 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:25.998528 29891 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:25.951953 29977 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.014719 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.000543 30128 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.992379 30190 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.015470 25787 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:26.000361 30176 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:19:26.003566 29970 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:26.019284 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 758) took 1235 ms (client timeout 1606 ms). Trace:
W20260709 12:19:26.023939 25787 rpcz_store.cc:269] 0709 12:19:24.784051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.007270 (+1223219us) service_pool.cc:224] Handling call
0709 12:19:26.019200 (+ 11930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.025511 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.027182 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2480) took 1118 ms. Trace:
I20260709 12:19:26.027676 25787 rpcz_store.cc:276] 0709 12:19:24.909207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.025107 (+1115900us) service_pool.cc:224] Handling call
0709 12:19:26.027090 (+  1983us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.029296 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.030336 30047 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:26.030825 25708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.011s	sys 0.015s
W20260709 12:19:26.031219 25708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.012s	sys 0.015s
I20260709 12:19:26.031422 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.032465 25787 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:26.033206 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2483) took 1062 ms. Trace:
I20260709 12:19:26.033659 25787 rpcz_store.cc:276] 0709 12:19:24.970714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.031028 (+1060314us) service_pool.cc:224] Handling call
0709 12:19:26.033139 (+  2111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.013964 30146 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:25.992761 30179 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:26.034662 25705 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.009s	sys 0.029s
I20260709 12:19:26.012761 29891 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.037240 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:26.038230 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:26.019800 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.018697 30022 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:26.038707 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.042348 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.035334 29970 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:26.046681 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.040145 25778 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:25.998443 30198 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:26.058077 25710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.015s	sys 0.017s
W20260709 12:19:26.079536 25710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.016s	sys 0.017s
I20260709 12:19:26.058427 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.056974 30047 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:26.084826 30184 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:26.085017 30047 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:26.085285 30184 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.085485 30047 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:26.085703 30184 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:26.088264 29966 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.991s	user 0.009s	sys 0.003s
I20260709 12:19:26.088810 29966 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:26.011219 30225 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.091360 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.092410 30225 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.092618 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.093159 26340 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:26.094278 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.094926 30209 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:26.056903 30158 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.096728 30158 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.098029 25787 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:26.098783 30158 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:26.098960 30226 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.099082 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.099785 26335 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:26.100818 29977 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:26.039364 25705 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.009s	sys 0.029s
I20260709 12:19:26.041738 29891 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:26.041980 30083 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.004s	sys 0.021s
I20260709 12:19:26.109853 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.110869 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.111296 30196 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:26.111974 30022 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:26.084596 30204 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.365s	user 0.004s	sys 0.006s
I20260709 12:19:26.057133 30140 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:26.125309 30022 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.126886 30179 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:26.116441 30198 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.081362 30164 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:26.118891 30083 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.005s	sys 0.021s
I20260709 12:19:26.043123 30146 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:26.119514 25704 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:26.120896 25708 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:26.121186 26338 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:26.123013 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.057884 30242 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:26.128403 30022 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.137310 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:26.134940 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.138900 26338 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:26.141101 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.129949 30164 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.139735 30209 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:19:26.142763 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 401031us
W20260709 12:19:26.148294 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.148360 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:26.148916 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.149560 25708 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:26.140969 30087 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:26.163079 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2668) took 2351 ms. Trace:
W20260709 12:19:26.152103 29962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.016s	sys 0.004s
I20260709 12:19:26.150027 30164 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:26.163482 25707 rpcz_store.cc:276] 0709 12:19:23.807867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.808502 (+   635us) service_pool.cc:224] Handling call
0709 12:19:23.808765 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.101638 (+292873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.158899 (+2057261us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54003c7dd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:26.159054 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671753984}
I20260709 12:19:26.169844 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.151523 30198 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:26.140095 30242 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.170907 25708 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
W20260709 12:19:26.142284 30213 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.011s	sys 0.008s
I20260709 12:19:26.158421 30087 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:26.143330 30192 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.179405 30146 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.173328 30190 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:26.191839 26341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 761) took 1996 ms (client timeout 1998 ms). Trace:
I20260709 12:19:26.190299 30047 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:26.154241 30150 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.016s	sys 0.023s
I20260709 12:19:26.254976 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.193812 25704 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:26.199250 30079 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:26.200565 30149 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:26.202273 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.204377 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 973) took 2251 ms. Trace:
I20260709 12:19:26.205420 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 753) took 1812 ms. Trace:
W20260709 12:19:26.206300 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2452) took 3598 ms (client timeout 3069 ms). Trace:
I20260709 12:19:26.214354 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2561) took 1493 ms. Trace:
I20260709 12:19:26.187949 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:26.168071 29962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.016s	sys 0.004s
W20260709 12:19:26.190560 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.225135 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.225551 30226 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.228497 30087 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:26.240294 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 252201us
W20260709 12:19:26.242636 26341 rpcz_store.cc:269] 0709 12:19:24.195406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.206043 (+ 10637us) service_pool.cc:224] Handling call
0709 12:19:26.191661 (+1985618us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:26.191784 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716961024}
W20260709 12:19:26.293951 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 51429.3us
I20260709 12:19:26.324054 25638 rpcz_store.cc:276] 0709 12:19:23.953354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.953534 (+   180us) service_pool.cc:224] Handling call
0709 12:19:23.953885 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.802432 (+848547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.204097 (+1401665us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54001d6fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:26.204316 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780615552}
I20260709 12:19:26.325610 25713 rpcz_store.cc:276] 0709 12:19:24.720806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.720987 (+   181us) service_pool.cc:224] Handling call
0709 12:19:26.214148 (+1493161us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400105c54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:26.214269 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409808768}
W20260709 12:19:26.323223 25786 rpcz_store.cc:269] 0709 12:19:22.608101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.292022 (+1683921us) service_pool.cc:224] Handling call
0709 12:19:26.206003 (+1913981us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b540011fb54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:26.206221 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2167210752}
I20260709 12:19:26.326395 25781 rpcz_store.cc:276] 0709 12:19:24.393355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.393648 (+   293us) service_pool.cc:224] Handling call
0709 12:19:24.394059 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:24.563108 (+169049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.563140 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:24.571997 (+  8857us) log.cc:844] Serialized 52 byte log entry
0709 12:19:24.795404 (+223407us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:24.795467 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.205005 (+1409538us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54001a77d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:26.205055 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.205104 (+    49us) raft_consensus.cc:1685] finished
0709 12:19:26.205120 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.205360 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9412,"raft.queue_time_us":4211,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":387487104,"thread_start_us":174584,"threads_started":1,"wal-append.queue_time_us":271545,"wal-append.run_cpu_time_us":2262,"wal-append.run_wall_time_us":1141755}
I20260709 12:19:26.329246 30149 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.215399 30127 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.322227 25708 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
W20260709 12:19:26.227314 30213 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.012s	sys 0.008s
I20260709 12:19:26.332713 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2568) took 1019 ms. Trace:
I20260709 12:19:26.335229 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.231647 29888 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:26.255638 30150 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.017s	sys 0.024s
I20260709 12:19:26.226862 30228 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.353703 25711 rpcz_store.cc:276] 0709 12:19:25.313513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.201971 (+888458us) service_pool.cc:224] Handling call
0709 12:19:26.332644 (+130673us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.366559 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.373811 26341 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:26.380292 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.254173 29993 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:26.357491 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:26.178704 30198 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.380918 25707 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:26.394423 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2486) took 1408 ms (client timeout 1657 ms). Trace:
I20260709 12:19:26.371037 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.367687 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:26.394845 25786 rpcz_store.cc:269] 0709 12:19:24.985690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.357153 (+1371463us) service_pool.cc:224] Handling call
0709 12:19:26.394365 (+ 37212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.330195 30171 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.380689 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.395813 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.396395 25786 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:26.403481 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.406714 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.381697 29888 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.407348 25713 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:26.408237 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.409384 30171 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.401157 29993 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.412032 30171 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.412864 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.413094 29993 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.413586 25713 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:26.414762 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.415442 25713 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:26.419298 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.420125 25711 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:26.421928 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:26.423058 26308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.068s	user 0.018s	sys 0.034s
W20260709 12:19:26.423740 26308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.018s	sys 0.035s
W20260709 12:19:26.424278 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 1.06891e+06us
I20260709 12:19:26.359905 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:26.426607 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 761) took 1641 ms (client timeout 2078 ms). Trace:
I20260709 12:19:26.426836 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:26.427722 25786 rpcz_store.cc:269] 0709 12:19:24.784940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.395553 (+1610613us) service_pool.cc:224] Handling call
0709 12:19:26.426463 (+ 30910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.432032 25636 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:26.417456 26341 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:26.428131 25635 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:26.379808 30216 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:26.398679 30212 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:26.444203 25706 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:26.422773 25708 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:26.441654 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:26.436229 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2473) took 1535 ms (client timeout 1852 ms). Trace:
I20260709 12:19:26.445930 25786 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:26.445842 25781 rpcz_store.cc:269] 0709 12:19:24.900436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.359643 (+1459207us) service_pool.cc:224] Handling call
0709 12:19:26.436151 (+ 76508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.434258 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.440901 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.448266 25636 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:26.449221 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.436373 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.452576 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2570) took 1137 ms. Trace:
I20260709 12:19:26.351159 30226 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.352211 30079 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.399101 30100 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.453995 25711 rpcz_store.cc:276] 0709 12:19:25.314760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.442449 (+1127689us) service_pool.cc:224] Handling call
0709 12:19:26.451624 (+  9175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.451973 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.452012 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:26.452110 (+    98us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:26.452178 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.452294 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.452490 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.484067 25707 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:26.350188 30146 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:26.486021 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 51843.5us
I20260709 12:19:26.450191 25781 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:26.486395 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.516757 25786 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:26.442224 30216 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:26.524395 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.012s	sys 0.052s
W20260709 12:19:26.543658 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.012s	sys 0.052s
W20260709 12:19:26.544303 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2475) took 1643 ms (client timeout 2089 ms). Trace:
I20260709 12:19:26.525702 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:26.544379 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 550146us
I20260709 12:19:26.529844 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.542171 25707 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:19:26.549528 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1956) took 1218 ms. Trace:
W20260709 12:19:26.545362 25786 rpcz_store.cc:269] 0709 12:19:24.900874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.441332 (+1540458us) service_pool.cc:224] Handling call
0709 12:19:26.544230 (+102898us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.549993 25781 rpcz_store.cc:276] 0709 12:19:25.331326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.448910 (+1117584us) service_pool.cc:224] Handling call
0709 12:19:26.549328 (+100418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.550777 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:26.551127 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.551783 25638 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:26.552160 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2481) took 1632 ms (client timeout 2144 ms). Trace:
W20260709 12:19:26.552651 25781 rpcz_store.cc:269] 0709 12:19:24.920259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.550810 (+1630551us) service_pool.cc:224] Handling call
0709 12:19:26.552090 (+  1280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.553680 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.544262 25713 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:26.555246 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2576) took 1069 ms. Trace:
I20260709 12:19:26.555610 25713 rpcz_store.cc:276] 0709 12:19:25.486058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.517691 (+1031633us) service_pool.cc:224] Handling call
0709 12:19:26.543907 (+ 26216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.554960 (+ 11053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.555182 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.566978 25781 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:26.568681 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2482) took 1601 ms (client timeout 2114 ms). Trace:
W20260709 12:19:26.570683 25781 rpcz_store.cc:269] 0709 12:19:24.967258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.553434 (+1586176us) service_pool.cc:224] Handling call
0709 12:19:26.568602 (+ 15168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:26.523197 30227 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:19:26.580446 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.574357 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.477439 29888 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.581815 25781 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:26.583472 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 53714.3us
I20260709 12:19:26.585789 25713 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:26.573582 30022 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:26.588299 25751 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50194 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 12:19:26.592235 30198 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.538271 30262 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:26.617031 26308 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.593534 30242 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.596921 30226 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.537876 30248 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:26.600528 30249 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.377s	user 0.000s	sys 0.001s
I20260709 12:19:26.602869 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1954) took 1272 ms. Trace:
I20260709 12:19:26.548635 30216 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.640025 26308 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.614895 30074 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:26.611295 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.620239 25632 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:19:26.624112 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.578375 30139 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:26.638278 25781 rpcz_store.cc:276] 0709 12:19:25.330803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.574081 (+1243278us) service_pool.cc:224] Handling call
0709 12:19:26.602791 (+ 28710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:26.641229 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:26.653738 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 328646us
I20260709 12:19:26.656502 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.658638 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.658927 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.630858 30262 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.664263 30262 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.665349 30226 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:26.660827 25713 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 5.
I20260709 12:19:26.649587 26343 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 5.
I20260709 12:19:26.674959 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.675873 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.678072 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:26.633004 30248 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.683943 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.684240 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.620205 30022 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.649519 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 983) took 2083 ms. Trace:
W20260709 12:19:26.680213 29891 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.561s	user 0.004s	sys 0.007s
I20260709 12:19:26.696825 25632 rpcz_store.cc:276] 0709 12:19:24.566553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.566692 (+   139us) service_pool.cc:224] Handling call
0709 12:19:26.619680 (+2052988us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b5800000c98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:19:26.619902 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.649221 (+ 29319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.649455 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2522496}
I20260709 12:19:26.666033 30242 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:26.652679 30139 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:26.656409 30247 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:26.660297 30192 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:26.642541 29663 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:19:26.678979 30175 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:26.679754 30047 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.698087 25638 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:26.683490 30074 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:26.717159 30261 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:19:26.718118 30264 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:19:26.728538 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2565) took 1422 ms. Trace:
W20260709 12:19:26.730751 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1950) took 2063 ms (client timeout 1705 ms). Trace:
I20260709 12:19:26.755599 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:26.755857 25778 rpcz_store.cc:269] 0709 12:19:24.667498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.223820 (+556322us) service_pool.cc:224] Handling call
0709 12:19:26.730456 (+1506636us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54002576d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:26.730661 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874558720}
I20260709 12:19:26.734192 30022 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.704491 29891 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.699615 30248 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:26.738282 30228 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:26.734226 30254 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.207s	user 0.001s	sys 0.001s
I20260709 12:19:26.645692 30253 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
W20260709 12:19:26.703715 30184 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.000s
I20260709 12:19:26.698745 30258 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:26.755656 25710 rpcz_store.cc:276] 0709 12:19:25.306406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.306970 (+   564us) service_pool.cc:224] Handling call
0709 12:19:25.323549 (+ 16579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:26.080076 (+756527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.080117 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:26.080654 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.516896 (+436242us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:26.516940 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.517009 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.728209 (+211200us) raft_consensus.cc:1685] finished
0709 12:19:26.728243 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.728450 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59495,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":1191948,"threads_started":2,"wal-append.queue_time_us":564624}
I20260709 12:19:26.704833 30242 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.737512 30198 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:26.768970 30210 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:26.747057 30086 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:19:26.712566 30139 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:26.613204 25751 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50194 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2476)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2497)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2498)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1957)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2493)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2500)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2492)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2494)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 31)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1952)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2471)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2478)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2479)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2484)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2485)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 762)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1961)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1963)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2488)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2489)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 770)
I20260709 12:19:26.712509 30247 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:26.780712 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.786933 30079 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.787894 30259 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:19:26.788444 30175 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:26.681741 30226 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.794888 25786 raft_consensus.cc:2368] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:26.795337 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 752) took 2399 ms. Trace:
I20260709 12:19:26.797317 30258 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:26.800143 30242 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.799981 29663 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:26.809891 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.797778 30155 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:26.792241 30228 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:26.800948 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 88680.7us
W20260709 12:19:26.818220 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 115532us
I20260709 12:19:26.819454 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.804464 30210 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:26.827517 30164 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.000s
I20260709 12:19:26.813313 30259 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:26.811053 30247 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:26.818275 30258 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:26.828244 29891 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:26.830315 30226 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.820477 29663 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:26.797434 30184 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.831557 30155 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.814200 30087 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.813407 25783 rpcz_store.cc:276] 0709 12:19:24.393028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.393154 (+   126us) service_pool.cc:224] Handling call
0709 12:19:24.421909 (+ 28755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.422325 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.422361 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:26.384515 (+1962154us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b2c0007afc8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:19:26.400072 (+ 15557us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.645603 (+245531us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:26.645672 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.645779 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.788869 (+143090us) raft_consensus.cc:1685] finished
0709 12:19:26.788895 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.792417 (+  3522us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73919,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6620,"spinlock_wait_cycles":20033920,"thread_start_us":200508,"threads_started":1,"wal-append.queue_time_us":340240}
I20260709 12:19:26.847388 26340 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:26.848037 30275 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.848913 30079 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.850174 25632 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:26.850365 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.851924 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.852859 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:26.915680 25630 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:26.854008 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.854030 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:26.855257 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:26.821913 29970 consensus_meta.cc:287] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.711s	user 0.012s	sys 0.000s
I20260709 12:19:26.856413 29799 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.857595 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:26.925194 25631 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:26.812700 30175 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:26.804661 30198 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.858319 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2680) took 1702 ms. Trace:
I20260709 12:19:26.860054 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 2690) took 1077 ms. Trace:
I20260709 12:19:26.860710 30228 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.867973 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:26.869330 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:26.920740 30149 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.005s
I20260709 12:19:26.870289 30192 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:26.880129 26342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 766) took 2664 ms (client timeout 2097 ms). Trace:
I20260709 12:19:26.871537 30262 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.870779 30022 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:26.892297 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 222776us
I20260709 12:19:26.905055 25751 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50982 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2497)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2498)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1957)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2493)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2500)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2492)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2494)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 31)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1952)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2471)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2478)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2479)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2484)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2485)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 762)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1961)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1963)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2488)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2489)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 771)
I20260709 12:19:26.887305 30259 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:26.910699 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.923713 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:26.929852 25637 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:26.887269 30164 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.909101 30221 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:26.956231 30221 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.833863 30216 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.958830 30238 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.959715 30216 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.940191 25704 rpcz_store.cc:276] 0709 12:19:25.782683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.037739 (+255056us) service_pool.cc:224] Handling call
0709 12:19:26.859854 (+822115us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b5400203354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:26.859984 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175880448}
I20260709 12:19:26.961010 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:26.961678 25704 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:26.962889 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.963521 25704 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:26.966815 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:26.968137 25677 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:26.970063 30213 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:26.970533 30213 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:26.970937 30213 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:26.934783 25712 rpcz_store.cc:276] 0709 12:19:25.156340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.163783 (+  7443us) service_pool.cc:224] Handling call
0709 12:19:25.164128 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.974625 (+810497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.857990 (+883365us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54002247d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:26.858244 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1862559104}
I20260709 12:19:26.973532 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.973663 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.975003 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.975136 25677 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:26.976064 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:26.976516 25677 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:26.978116 30260 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.450s	user 0.001s	sys 0.000s
I20260709 12:19:26.916033 30275 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.952636 30192 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:26.980356 30275 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:26.980911 30192 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:26.954811 30022 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.981894 30087 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:26.983011 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:26.983217 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2579) took 1014 ms. Trace:
I20260709 12:19:26.983933 25712 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:26.984194 25783 raft_consensus.cc:2410] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 12:19:26.941383 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74019.7us
W20260709 12:19:26.942605 26342 rpcz_store.cc:269] 0709 12:19:24.210718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.211983 (+  1265us) service_pool.cc:224] Handling call
0709 12:19:26.874182 (+2662199us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54004812d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:26.874298 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551693056}
W20260709 12:19:26.943584 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 75711.2us
I20260709 12:19:26.954962 25710 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:26.959431 30238 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.987975 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.988171 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.990518 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.941936 25632 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:26.993983 26341 raft_consensus.cc:2368] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:19:26.994953 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.983685 25706 rpcz_store.cc:276] 0709 12:19:25.968557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.254589 (+286032us) service_pool.cc:224] Handling call
0709 12:19:26.983029 (+728440us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400073ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:26.983157 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272946432}
I20260709 12:19:26.986373 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:26.996960 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:26.997467 25630 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:26.999087 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:26.998411 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:26.993744 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.002085 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.002856 25629 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:27.003579 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.001467 25630 raft_consensus.cc:2376] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10. Re-sending same reply.
I20260709 12:19:27.000211 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:26.949056 30047 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:26.917164 30079 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:26.948041 29799 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:27.007874 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.007891 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.008123 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.004146 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:26.924942 29970 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.012518 25750 leader_election.cc:400] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:19:27.013557 26338 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:27.007916 30087 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:27.021955 25750 leader_election.cc:403] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:26.999209 30226 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:27.012404 29901 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:27.018896 25632 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:27.019558 25704 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:27.018254 26341 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:27.024248 29964 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:19:27.024858 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:26.946686 30279 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:19:27.009168 30265 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:27.036005 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:27.040714 25677 leader_election.cc:400] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:27.005550 30228 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:27.005920 29891 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:26.949754 30149 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.038617 25638 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:26.953214 30262 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.041158 25677 leader_election.cc:403] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:19:27.042909 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2698) took 1216 ms. Trace:
I20260709 12:19:26.983007 30175 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.043962 25710 raft_consensus.cc:2410] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:27.017096 29799 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:27.056558 25704 rpcz_store.cc:276] 0709 12:19:25.826585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.037942 (+1211357us) service_pool.cc:224] Handling call
0709 12:19:27.038334 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.042479 (+  4145us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:27.042530 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:27.042581 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:27.042617 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.042694 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.042848 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3731}
I20260709 12:19:27.027470 25751 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50966 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 759)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2497)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2498)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1957)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2493)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2500)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2492)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2494)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 31)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1952)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2471)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2478)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2479)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2484)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2485)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 762)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1961)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1963)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2488)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2489)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 771)
I20260709 12:19:27.044296 30228 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.308904 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2564) took 2231 ms. Trace:
W20260709 12:19:27.211450 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.947s (SENT) [suppressed 27 similar messages]
I20260709 12:19:26.994454 29663 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.323622 29663 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:27.211831 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2578) took 1243 ms. Trace:
I20260709 12:19:27.082710 29799 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:27.035951 29984 raft_consensus.cc:2764] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:19:27.012216 30192 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:19:27.212972 30184 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:27.213714 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.213569 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2696) took 1429 ms. Trace:
W20260709 12:19:27.214973 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 155929us
I20260709 12:19:27.214684 30175 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:27.218063 30216 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:27.219414 30262 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:27.219525 30198 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.211596 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:27.053859 29970 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:27.037850 30074 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:27.267984 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.356541 25782 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:27.202275 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.238937 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2476) took 2336 ms (client timeout 2539 ms). Trace:
I20260709 12:19:27.243904 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:27.358672 25783 rpcz_store.cc:269] 0709 12:19:24.902929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.912256 (+2009327us) service_pool.cc:224] Handling call
0709 12:19:27.238631 (+326375us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b54000870d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.238829 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73295232}
I20260709 12:19:27.359670 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.359965 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:27.360419 25629 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
W20260709 12:19:27.360811 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2497) took 1389 ms (client timeout 1720 ms). Trace:
W20260709 12:19:27.361179 25783 rpcz_store.cc:269] 0709 12:19:25.971755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.359684 (+1387929us) service_pool.cc:224] Handling call
0709 12:19:27.360752 (+  1068us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.361627 25629 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:27.362043 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.245493 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:27.242416 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.233436 30221 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:27.236869 30269 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.530s	user 0.001s	sys 0.000s
I20260709 12:19:27.287853 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.229815 29964 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10.
I20260709 12:19:27.268276 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.226874 30226 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:27.317132 25710 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:27.273512 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:27.225884 30265 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.275883 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:27.265228 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1953) took 2596 ms (client timeout 1691 ms). Trace:
I20260709 12:19:27.369733 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.274888 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2577) took 1666 ms. Trace:
W20260709 12:19:27.370276 25785 rpcz_store.cc:269] 0709 12:19:24.668329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.838027 (+169698us) service_pool.cc:224] Handling call
0709 12:19:27.264952 (+2426925us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400099a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.265100 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282359168}
W20260709 12:19:27.319478 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 108063us
I20260709 12:19:27.370690 25707 rpcz_store.cc:276] 0709 12:19:25.608740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.521697 (+912957us) service_pool.cc:224] Handling call
0709 12:19:26.522040 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.549480 (+ 27440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.274553 (+725073us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b54002ec2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:27.274821 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45044608}
I20260709 12:19:27.369736 25781 raft_consensus.cc:2368] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
W20260709 12:19:27.372114 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1959) took 1569 ms (client timeout 1621 ms). Trace:
I20260709 12:19:27.364039 25751 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50966 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1957)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2493)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2500)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2492)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2494)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 31)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1952)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2471)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2477)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2478)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2479)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2484)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2485)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 762)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1961)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1963)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2488)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2489)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 771)
W20260709 12:19:27.372457 25781 rpcz_store.cc:269] 0709 12:19:25.803057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.273140 (+1470083us) service_pool.cc:224] Handling call
0709 12:19:27.372058 (+ 98918us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.319021 25705 rpcz_store.cc:276] 0709 12:19:25.077729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.077870 (+   141us) service_pool.cc:224] Handling call
0709 12:19:25.088487 (+ 10617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.311872 (+223385us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:26.102833 (+790961us) log.cc:844] Serialized 14 byte log entry
0709 12:19:26.897563 (+794730us) log.cc:844] Serialized 14 byte log entry
0709 12:19:26.897876 (+   313us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:26.898586 (+   710us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.902833 (+  4247us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:26.903553 (+   720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.904703 (+  1150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:27.308668 (+403965us) raft_consensus.cc:1685] finished
0709 12:19:27.308694 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.308836 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19522,"prepare.queue_time_us":809756,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":5203,"raft.queue_time_us":81760,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":42077,"thread_start_us":1725820,"threads_started":4,"wal-append.queue_time_us":512024}
I20260709 12:19:27.373324 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.329646 25704 rpcz_store.cc:276] 0709 12:19:25.968249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.059880 (+1091631us) service_pool.cc:224] Handling call
0709 12:19:27.211061 (+151181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.211280 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:27.211308 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:27.211354 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:27.211385 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.211583 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.211767 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:27.314709 30278 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.327s	user 0.000s	sys 0.006s
W20260709 12:19:27.327776 30242 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.002s
I20260709 12:19:27.345073 25713 rpcz_store.cc:276] 0709 12:19:25.784235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.680609 (+896374us) service_pool.cc:224] Handling call
0709 12:19:26.680918 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.681123 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:26.905061 (+223938us) log.cc:844] Serialized 14 byte log entry
0709 12:19:27.213215 (+308154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:27.213275 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:27.213324 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.213399 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.213514 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":324850,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":1454,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":516211,"threads_started":2,"wal-append.queue_time_us":365568}
I20260709 12:19:27.313593 30228 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:27.354192 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 290796us
I20260709 12:19:27.385154 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.385861 26341 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
W20260709 12:19:27.362998 25782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 759) took 2578 ms (client timeout 2789 ms). Trace:
I20260709 12:19:27.229370 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1933) took 4550 ms. Trace:
I20260709 12:19:27.362983 25629 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:27.308830 30139 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.364542 26338 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:27.364981 25783 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:27.311403 30248 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.344053 30210 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.373895 25781 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
W20260709 12:19:27.392155 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1957) took 2060 ms (client timeout 2522 ms). Trace:
W20260709 12:19:27.380328 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.382953 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.392509 25781 rpcz_store.cc:269] 0709 12:19:25.331612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.373093 (+2041481us) service_pool.cc:224] Handling call
0709 12:19:27.392096 (+ 19003us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.376452 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2580) took 1407 ms. Trace:
W20260709 12:19:27.388504 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2498) took 1390 ms (client timeout 1852 ms). Trace:
W20260709 12:19:27.393276 25783 rpcz_store.cc:269] 0709 12:19:25.998518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.361809 (+1363291us) service_pool.cc:224] Handling call
0709 12:19:27.388448 (+ 26639us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.393375 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:27.375312 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1960) took 1572 ms (client timeout 1611 ms). Trace:
W20260709 12:19:27.386655 25782 rpcz_store.cc:269] 0709 12:19:24.784306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.267703 (+2483397us) service_pool.cc:224] Handling call
0709 12:19:27.362938 (+ 95235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.394086 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:27.386868 25780 rpcz_store.cc:276] 0709 12:19:22.679289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.822340 (+143051us) service_pool.cc:224] Handling call
0709 12:19:24.292606 (+1470266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.232863 (+940257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.228255 (+1995392us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54002b4654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:27.229134 (+   879us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002b4650. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.229282 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5515413248}
I20260709 12:19:27.389869 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.362668 30288 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:27.394071 25710 rpcz_store.cc:276] 0709 12:19:25.968859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.064827 (+1095968us) service_pool.cc:224] Handling call
0709 12:19:27.310206 (+245379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.376221 (+ 66015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.376400 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:27.395093 25778 rpcz_store.cc:269] 0709 12:19:25.803343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.275639 (+1472296us) service_pool.cc:224] Handling call
0709 12:19:27.375256 (+ 99617us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.400977 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.394939 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.403139 25784 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:19:27.399673 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.394071 25781 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:27.401486 25778 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:27.411705 26338 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:27.414966 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:27.415457 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.415701 25778 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:27.414422 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2493) took 1443 ms. Trace:
I20260709 12:19:27.395993 25629 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:27.416267 25781 rpcz_store.cc:276] 0709 12:19:25.970696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.393156 (+1422460us) service_pool.cc:224] Handling call
0709 12:19:27.414316 (+ 21160us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:27.402911 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2455) took 4793 ms (client timeout 3030 ms). Trace:
I20260709 12:19:27.417272 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.417975 25781 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:27.418864 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.419822 25782 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:27.420553 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1958) took 2088 ms. Trace:
I20260709 12:19:27.421025 25782 rpcz_store.cc:276] 0709 12:19:25.331900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.414715 (+2082815us) service_pool.cc:224] Handling call
0709 12:19:27.420483 (+  5768us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.422097 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.415910 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:27.423424 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 764) took 2009 ms. Trace:
I20260709 12:19:27.424058 25782 rpcz_store.cc:276] 0709 12:19:25.414059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.421823 (+2007764us) service_pool.cc:224] Handling call
0709 12:19:27.423033 (+  1210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.425196 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.425422 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.419426 25781 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:27.417104 25779 rpcz_store.cc:269] 0709 12:19:22.609262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.216687 (+1607425us) service_pool.cc:224] Handling call
0709 12:19:27.402664 (+3185977us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54001a81d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.402835 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2034189184}
I20260709 12:19:27.427449 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1962) took 1623 ms. Trace:
I20260709 12:19:27.426749 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.423631 25783 raft_consensus.cc:2376] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9. Re-sending same reply.
I20260709 12:19:27.428124 25781 rpcz_store.cc:276] 0709 12:19:25.803839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.418664 (+1614825us) service_pool.cc:224] Handling call
0709 12:19:27.427398 (+  8734us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.429405 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2500) took 1309 ms. Trace:
I20260709 12:19:27.429731 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.429940 25783 rpcz_store.cc:276] 0709 12:19:26.120256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.393985 (+1273729us) service_pool.cc:224] Handling call
0709 12:19:27.429308 (+ 35323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.430923 25781 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:27.386063 30022 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.432049 26335 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:27.436929 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.431636 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 767) took 1049 ms. Trace:
I20260709 12:19:27.438755 25781 rpcz_store.cc:276] 0709 12:19:26.382751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.429497 (+1046746us) service_pool.cc:224] Handling call
0709 12:19:27.431580 (+  2083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.426762 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.433565 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.440517 25780 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:27.441059 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 765) took 1089 ms. Trace:
I20260709 12:19:27.441509 25780 rpcz_store.cc:276] 0709 12:19:26.351939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.402224 (+1050285us) service_pool.cc:224] Handling call
0709 12:19:27.441005 (+ 38781us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.394212 30120 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.433857 25782 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:27.467512 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 766) took 1095 ms. Trace:
I20260709 12:19:27.467893 25782 rpcz_store.cc:276] 0709 12:19:26.371681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.426529 (+1054848us) service_pool.cc:224] Handling call
0709 12:19:27.467451 (+ 40922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.468856 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.382265 30265 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.439738 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.471729 25781 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:27.472258 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2495) took 1501 ms. Trace:
I20260709 12:19:27.472631 25781 rpcz_store.cc:276] 0709 12:19:25.971257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.439504 (+1468247us) service_pool.cc:224] Handling call
0709 12:19:27.472182 (+ 32678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.473783 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.444080 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.447444 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.449905 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.450126 30248 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.451597 30139 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.428097 30288 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:27.455071 30149 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.457293 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.484072 25629 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:27.459908 25676 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50966 dropped due to backpressure: service queue is full with 50 items
W20260709 12:19:27.463766 30258 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.002s
I20260709 12:19:27.463973 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:27.469499 25782 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:27.442942 26338 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:27.468741 26336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 905) took 1909 ms (client timeout 1947 ms). Trace:
W20260709 12:19:27.506810 26336 rpcz_store.cc:269] 0709 12:19:25.559381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.560305 (+   924us) service_pool.cc:224] Handling call
0709 12:19:27.468595 (+1908290us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400498ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.468682 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":855777664}
I20260709 12:19:27.507182 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.508082 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:27.508872 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.509684 25782 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:27.510053 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2496) took 1538 ms. Trace:
I20260709 12:19:27.510358 25782 rpcz_store.cc:276] 0709 12:19:25.971489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.508623 (+1537134us) service_pool.cc:224] Handling call
0709 12:19:27.510001 (+  1378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.511180 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1948) took 2849 ms. Trace:
I20260709 12:19:27.511274 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:27.435916 30247 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
W20260709 12:19:27.512315 30283 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.518s	user 0.001s	sys 0.000s
I20260709 12:19:27.512835 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.512629 30247 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.513475 25782 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:27.514302 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:27.475237 25779 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:27.515595 30247 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.516582 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.517079 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 695) took 2932 ms. Trace:
I20260709 12:19:27.517871 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.518949 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.520635 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.521787 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.482975 30087 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:27.523057 30258 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.525458 30258 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.527711 30265 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:27.529798 25780 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:27.515437 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.421656 30164 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.479341 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.428880 30079 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.480978 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.452158 29799 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.389102 30242 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.530761 25637 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:27.531378 25782 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:27.543427 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2492) took 1587 ms. Trace:
I20260709 12:19:27.543792 25782 rpcz_store.cc:276] 0709 12:19:25.956181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.515149 (+1558968us) service_pool.cc:224] Handling call
0709 12:19:27.543369 (+ 28220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15104}
I20260709 12:19:27.518775 30197 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:27.505884 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:27.477133 25781 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:27.515501 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.517494 25709 rpcz_store.cc:276] 0709 12:19:24.585023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.589430 (+  4407us) service_pool.cc:224] Handling call
0709 12:19:24.616220 (+ 26790us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:24.653485 (+ 37265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.653531 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:24.654219 (+   688us) log.cc:844] Serialized 52 byte log entry
0709 12:19:25.795086 (+1140867us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b1800452980. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:26.740316 (+945230us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b1800452980. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:19:26.797514 (+ 57198us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:26.797583 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.799906 (+  2323us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:27.511688 (+711782us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:27.516833 (+  5145us) raft_consensus.cc:1685] finished
0709 12:19:27.516858 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.517015 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35533,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":37249,"spinlock_wait_cycles":45311488,"thread_start_us":56894,"threads_started":1,"wal-append.queue_time_us":2227182}
I20260709 12:19:27.511842 25784 rpcz_store.cc:276] 0709 12:19:24.662208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.664775 (+  2567us) service_pool.cc:224] Handling call
0709 12:19:27.402826 (+2738051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:27.413868 (+ 11042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.510974 (+ 97106us) spinlock_profiling.cc:243] Waited 2.74 s on lock 0x7b54001a81d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.511114 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6296072064}
I20260709 12:19:27.534427 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:27.550457 26338 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:27.551164 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.552824 26341 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:27.552877 25629 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:27.554839 26336 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:27.555490 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:27.555975 26336 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:27.541461 25783 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 6: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:27.538244 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:27.551059 25779 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:27.556591 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.558943 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.559639 25635 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:27.558974 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:27.476266 30120 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.563786 25778 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:27.563817 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 103939us
W20260709 12:19:27.565495 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2474) took 2664 ms (client timeout 1720 ms). Trace:
I20260709 12:19:27.563102 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:27.562027 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:27.565913 25787 rpcz_store.cc:269] 0709 12:19:24.900657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.037940 (+1137283us) service_pool.cc:224] Handling call
0709 12:19:27.565329 (+1527389us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b54000b04d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.565431 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111840000}
I20260709 12:19:27.535100 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 992) took 1085 ms. Trace:
I20260709 12:19:27.569926 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.570569 25631 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:27.569953 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:27.574443 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:27.577683 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:27.579162 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:27.580890 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:27.572330 30213 consensus_meta.cc:287] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.005s
I20260709 12:19:27.567936 25636 rpcz_store.cc:276] 0709 12:19:26.450031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.450574 (+   543us) service_pool.cc:224] Handling call
0709 12:19:26.450979 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:26.451355 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.451386 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:26.485076 (+ 33690us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.822246 (+337170us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:26.823033 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.823117 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:27.486282 (+663165us) raft_consensus.cc:1685] finished
0709 12:19:27.534894 (+ 48612us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.535040 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135387,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":242939,"threads_started":1,"wal-append.queue_time_us":375918}
I20260709 12:19:27.582370 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:27.598740 30213 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.599428 25710 raft_consensus.cc:2410] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:27.544682 30151 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:19:27.601207 30213 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.588740 30061 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:27.579891 30259 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.006s
I20260709 12:19:27.601974 30061 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.602790 30259 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.605417 30259 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.556699 30197 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:27.561028 30164 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:27.611888 25709 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:27.613999 26340 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:27.628356 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1964) took 1055 ms. Trace:
I20260709 12:19:27.633984 25778 rpcz_store.cc:276] 0709 12:19:26.573291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.436673 (+863382us) service_pool.cc:224] Handling call
0709 12:19:27.628164 (+191491us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540011f154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.628293 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290248448}
I20260709 12:19:27.629622 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2685) took 1866 ms. Trace:
I20260709 12:19:27.636408 25711 rpcz_store.cc:276] 0709 12:19:25.763172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.523827 (+760655us) service_pool.cc:224] Handling call
0709 12:19:26.984259 (+460432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.984444 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.984464 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:26.984500 (+    36us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:26.984530 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.984579 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.629449 (+644870us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400073ad0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.629557 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058226432}
I20260709 12:19:27.569700 30056 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:27.568993 30226 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:27.635524 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:27.643788 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.644773 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:27.639446 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60366.8us
I20260709 12:19:27.646673 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:27.638515 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50714.8us
I20260709 12:19:27.639113 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.572810 30221 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:27.648993 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.649269 30221 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.649613 25634 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:27.649863 30221 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:27.657456 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2494) took 1686 ms. Trace:
I20260709 12:19:27.658046 25778 rpcz_store.cc:276] 0709 12:19:25.970997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.644576 (+1673579us) service_pool.cc:224] Handling call
0709 12:19:27.657376 (+ 12800us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.659021 25634 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:27.661176 25603 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:27.578444 30210 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:27.663568 25786 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1955) took 2332 ms (client timeout 1710 ms). Trace:
W20260709 12:19:27.667873 25786 rpcz_store.cc:269] 0709 12:19:25.331086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.550504 (+1219418us) service_pool.cc:224] Handling call
0709 12:19:27.663330 (+1112826us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540035a2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:27.663492 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559498624}
I20260709 12:19:27.662360 25603 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:27.577844 30258 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:27.668435 25778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:27.670297 25778 raft_consensus.cc:606] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:27.670758 25778 raft_consensus.cc:614] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:27.671317 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36022 (request call id 31) took 2887 ms. Trace:
I20260709 12:19:27.671680 25778 rpcz_store.cc:276] 0709 12:19:24.783748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.668302 (+2884554us) service_pool.cc:224] Handling call
0709 12:19:27.671244 (+  2942us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:27.675395 25601 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.720s (SENT)
I20260709 12:19:27.676028 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:27.584071 29901 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:27.615252 30198 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:27.680437 25635 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:27.683627 25676 leader_election.cc:400] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:27.690822 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:27.691509 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:27.692128 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:27.690310 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:27.695353 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:27.631323 29987 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:27.705930 25752 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:27.642316 30100 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:27.725497 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:27.667064 30120 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.667929 30056 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:27.727797 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:27.739312 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:27.740007 25705 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:27.740439 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:27.682583 29901 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:27.732036 25710 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:27.730029 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:27.726711 25634 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:27.680302 30079 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:27.697177 30164 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:27.750094 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:27.736269 25704 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:27.725487 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 792) took 1627 ms. Trace:
I20260709 12:19:27.756373 29970 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:27.735441 25635 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:19:27.723955 29987 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.779723 29987 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.751407 25783 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:27.782799 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1970) took 1196 ms. Trace:
I20260709 12:19:27.783123 25783 rpcz_store.cc:276] 0709 12:19:26.586151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.685210 (+1099059us) service_pool.cc:224] Handling call
0709 12:19:27.782735 (+ 97525us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:27.632251 30128 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:19:27.785018 30128 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:27.785362 30128 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:27.736819 30100 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.786149 30128 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.787333 30128 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:27.788265 30100 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:27.788939 30151 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:27.789386 25635 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: [7.6-10.7]
I20260709 12:19:27.725670 30258 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:27.741830 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79534.8us
I20260709 12:19:27.813835 25603 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:27.756282 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 80943.9us
I20260709 12:19:27.725565 30279 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.692816 29962 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:19:27.739094 30265 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:27.760489 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 175181us
I20260709 12:19:27.820271 30265 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:27.820719 30265 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.730906 30274 raft_consensus.cc:2764] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:19:27.759878 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64631.8us
I20260709 12:19:27.823553 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:27.738867 30061 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.751206 30087 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:27.752693 30120 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:27.824872 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:27.825196 30120 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:27.825696 30120 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:27.693184 30210 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.751392 29901 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:27.828249 30210 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.828868 30210 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:27.829227 30210 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.829569 30210 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:27.701448 30213 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:27.770112 26339 rpcz_store.cc:276] 0709 12:19:26.098386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.098541 (+   155us) service_pool.cc:224] Handling call
0709 12:19:26.098954 (+   413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:26.129828 (+ 30874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.129865 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:26.131968 (+  2103us) log.cc:844] Serialized 52 byte log entry
0709 12:19:26.542833 (+410865us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:26.542879 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.697545 (+1154666us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b540061d0d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:27.697610 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:27.697661 (+    51us) raft_consensus.cc:1685] finished
0709 12:19:27.697675 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.725390 (+ 27715us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92,"raft.queue_time_us":447,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":66249984,"thread_start_us":408305,"threads_started":1,"wal-append.queue_time_us":567767}
I20260709 12:19:27.830701 30213 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:27.831187 30213 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:27.772918 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:27.834872 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62051.7us
I20260709 12:19:27.835788 29964 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:27.835456 30164 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.775177 30022 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:27.773777 29970 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:27.757012 30242 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:27.794546 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.792220 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 993) took 1339 ms. Trace:
I20260709 12:19:27.807688 29799 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.819446 30192 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:27.820601 25677 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:27.706847 25631 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:27.850181 25633 rpcz_store.cc:276] 0709 12:19:26.452883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.453051 (+   168us) service_pool.cc:224] Handling call
0709 12:19:26.521794 (+ 68743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.543270 (+ 21476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.543298 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:26.543335 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:26.543362 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.791873 (+1248511us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b540020b7d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:27.791943 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.792136 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48949632}
I20260709 12:19:27.837239 25676 leader_election.cc:400] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:19:27.778755 30056 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.702644 30288 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.838954 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:27.864837 26339 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
W20260709 12:19:27.842015 25601 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.852s (SENT)
I20260709 12:19:27.802551 30258 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:27.866096 30258 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:27.847297 30299 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:27.867630 30299 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:27.868019 30299 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:27.834228 30155 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:27.815140 30079 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:27.857276 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:27.859810 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:27.880813 29964 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:27.849246 29970 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:27.760757 30197 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.887627 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.859166 30022 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:19:27.905691 30128 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:27.907298 30192 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:27.908083 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.915804 25635 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:27.854176 30279 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.916747 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.916608 29987 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.917492 25635 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:27.918092 30279 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:27.870635 30295 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:19:27.882172 29964 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.884840 30274 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:19:27.919395 30213 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.920956 29901 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.894155 30155 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:27.922106 29987 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.922174 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:27.926362 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.926754 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1951) took 3259 ms. Trace:
I20260709 12:19:27.956329 25786 rpcz_store.cc:276] 0709 12:19:24.667794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.690693 (+3022899us) service_pool.cc:224] Handling call
0709 12:19:27.691036 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:27.691433 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:27.691462 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:27.691951 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:19:27.756381 (+ 64430us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:27.887526 (+131145us) log.cc:844] Serialized 14 byte log entry
0709 12:19:27.926326 (+ 38800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.926435 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:27.926520 (+    85us) raft_consensus.cc:1685] finished
0709 12:19:27.926542 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.926686 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38218,"prepare.queue_time_us":186289,"raft.queue_time_us":39305,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":30471,"thread_start_us":185067,"threads_started":2,"wal-append.queue_time_us":210527}
I20260709 12:19:27.927083 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:27.958973 25704 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:27.860503 30243 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:27.927675 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:27.928644 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:27.942736 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:27.967334 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 94174.2us
I20260709 12:19:27.946957 25631 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:27.909989 30221 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:27.891806 30226 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.982756 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:27.983721 25630 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.950598 30151 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:27.952610 30295 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:19:27.985306 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 149186us
I20260709 12:19:27.995522 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2471) took 3095 ms. Trace:
I20260709 12:19:27.995661 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:27.996232 25783 rpcz_store.cc:276] 0709 12:19:24.899942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.783854 (+2883912us) service_pool.cc:224] Handling call
0709 12:19:27.801052 (+ 17198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.801314 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:27.920813 (+119499us) log.cc:844] Serialized 14 byte log entry
0709 12:19:27.995015 (+ 74202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:27.995121 (+   106us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:27.995189 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.995277 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.995449 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":167719,"thread_start_us":188782,"threads_started":2}
I20260709 12:19:27.955739 30155 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:27.995142 30295 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:28.009681 30155 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:28.010435 30295 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.011703 30295 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:28.013887 25630 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:28.014804 30306 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:28.020032 29901 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.021710 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.022712 30221 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:27.995850 30226 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.025717 30146 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.025923 30226 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:28.028144 30128 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:19:28.035810 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:27.999074 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:28.001374 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2477) took 3098 ms. Trace:
I20260709 12:19:28.041193 25783 rpcz_store.cc:276] 0709 12:19:24.903184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.998104 (+3094920us) service_pool.cc:224] Handling call
0709 12:19:28.000575 (+  2471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.000825 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.000869 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:28.000929 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:28.001036 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.001117 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.001259 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.041635 25752 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:28.043344 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:28.044737 25752 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:28.049953 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 192783us
I20260709 12:19:28.005628 30299 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:27.982529 29987 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:28.050868 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:28.057801 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:28.059723 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:28.062877 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 209850us
I20260709 12:19:28.067312 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:28.068331 26339 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:19:28.047886 30228 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.644s	user 0.016s	sys 0.001s
W20260709 12:19:28.069324 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.069224 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:28.072476 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:28.073344 26336 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:28.073508 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:28.038719 30061 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:28.077080 25783 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:28.077962 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2479) took 3173 ms. Trace:
I20260709 12:19:28.078392 25783 rpcz_store.cc:276] 0709 12:19:24.904139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.042295 (+3138156us) service_pool.cc:224] Handling call
0709 12:19:28.065642 (+ 23347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.077606 (+ 11964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.077887 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.053362 30022 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:28.053534 30306 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:28.082391 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:28.054527 30289 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:28.083468 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:28.057528 29987 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.063380 30128 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:28.089828 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:28.067179 30226 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:28.084178 30061 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.090618 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:28.030349 30210 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.093164 26339 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:28.069092 30221 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:28.094281 25783 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:28.106830 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2484) took 3129 ms. Trace:
I20260709 12:19:28.091470 30306 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:28.087242 30308 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:28.091187 30243 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:28.090358 29987 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:28.107231 25783 rpcz_store.cc:276] 0709 12:19:24.977237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.079530 (+3102293us) service_pool.cc:224] Handling call
0709 12:19:28.094046 (+ 14516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.106449 (+ 12403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.106760 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.102061 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:28.103953 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:28.144764 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.145947 25781 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:28.081959 29984 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:19:28.104614 30061 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:28.111732 30175 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.758s	user 0.016s	sys 0.002s
I20260709 12:19:28.085116 30228 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.092289 30274 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:19:28.092180 30128 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:28.109004 30288 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:28.114743 30299 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:28.119308 30213 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.119938 30306 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:28.128381 30210 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.131788 25704 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:19:28.136768 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:28.139184 30263 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.474s	user 0.004s	sys 0.000s
I20260709 12:19:28.140247 30238 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.140172 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.097856 30226 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.103811 30022 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:28.153141 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49263.3us
I20260709 12:19:28.142426 30308 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:28.085546 30120 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.106273 30221 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.093303 30197 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:19:28.154973 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:28.107908 30056 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:28.196487 30221 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:28.192507 30226 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:28.221724 25636 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:28.223171 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:28.171932 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:28.174469 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.224570 25783 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:28.224967 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:28.180697 30311 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:19:28.184265 30310 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:28.110347 30289 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:19:28.143862 30243 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:28.188113 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 91961.2us
I20260709 12:19:28.189095 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:28.167014 30128 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:28.328933 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.329952 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:28.331727 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:28.192523 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:28.197253 25633 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:28.206804 30197 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:28.208189 29970 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.209102 30056 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:28.210017 25601 leader_election.cc:336] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 2.089s (SENT)
I20260709 12:19:28.367976 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:28.156028 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.214745 30221 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:28.219872 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 768) took 1826 ms (client timeout 1703 ms). Trace:
I20260709 12:19:28.154399 30047 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:28.195895 30308 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:28.227175 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:28.227854 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 775) took 4004 ms. Trace:
I20260709 12:19:28.229523 25710 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:28.245090 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.256000 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 916) took 1876 ms. Trace:
I20260709 12:19:28.260268 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1971) took 1673 ms. Trace:
I20260709 12:19:28.265434 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:28.269975 25638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2348) took 1420 ms (client timeout 1826 ms). Trace:
I20260709 12:19:28.280086 25677 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:28.282778 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:28.226547 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2472) took 3326 ms. Trace:
I20260709 12:19:28.284477 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.230919 30120 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.357558 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:28.344977 26307 connection.cc:256] RPC call timeout handler was delayed by -1.2754983720001292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27890000000000004s for a response
W20260709 12:19:28.426842 25638 rpcz_store.cc:269] 0709 12:19:26.849207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.851651 (+  2444us) service_pool.cc:224] Handling call
0709 12:19:28.269757 (+1418106us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54001d79d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:28.269874 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283085568}
I20260709 12:19:28.428833 25787 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:28.432256 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:28.432852 25707 raft_consensus.cc:2376] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11. Re-sending same reply.
I20260709 12:19:28.432479 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:28.366784 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 87342.7us
I20260709 12:19:28.243690 30243 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:28.372224 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:28.232584 30228 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:28.231112 30140 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.263469 30224 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.245743 29962 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.143s
I20260709 12:19:28.244226 30265 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:28.371942 25785 rpcz_store.cc:269] 0709 12:19:26.393729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.219638 (+1825909us) service_pool.cc:211] Skipping call since client already timed out
0709 12:19:28.219778 (+   140us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:19:28.333191 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 140764us
I20260709 12:19:28.426753 25786 rpcz_store.cc:276] 0709 12:19:24.900212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.958557 (+3058345us) service_pool.cc:224] Handling call
0709 12:19:27.961002 (+  2445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:27.961432 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:27.961462 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:27.973110 (+ 11648us) log.cc:844] Serialized 26 byte log entry
0709 12:19:28.165327 (+192217us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:28.165370 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.165451 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:28.226310 (+ 60859us) raft_consensus.cc:1685] finished
0709 12:19:28.226332 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.226470 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":177,"raft.queue_time_us":3054,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":11738,"thread_start_us":157069,"threads_started":1,"wal-append.queue_time_us":212581}
I20260709 12:19:28.434055 25711 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:19:28.434803 25784 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:28.426788 25782 rpcz_store.cc:276] 0709 12:19:26.586510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.550921 (+964411us) service_pool.cc:224] Handling call
0709 12:19:28.260043 (+709122us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54001252d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:28.260192 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224084736}
I20260709 12:19:28.365302 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1076) took 1414 ms. Trace:
I20260709 12:19:28.437460 25603 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:28.354453 30197 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:28.247155 30061 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.579144 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:28.580005 25785 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:28.580564 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.581239 25783 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:28.255476 30258 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.232496 30155 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.246407 29987 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.373323 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:28.428462 26337 rpcz_store.cc:276] 0709 12:19:24.223876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.903999 (+680123us) service_pool.cc:224] Handling call
0709 12:19:24.968257 (+ 64258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.909913 (+941656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.909948 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.931692 (+ 21744us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.332272 (+400580us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:26.332323 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.863020 (+530697us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b5400498254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:26.863085 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:27.987689 (+1124604us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:28.227623 (+239934us) raft_consensus.cc:1685] finished
0709 12:19:28.227647 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.227773 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15344,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4645,"spinlock_wait_cycles":2174835328,"thread_start_us":334901,"threads_started":1,"wal-append.queue_time_us":676555}
I20260709 12:19:28.435312 26335 rpcz_store.cc:276] 0709 12:19:26.380017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.684004 (+303987us) service_pool.cc:224] Handling call
0709 12:19:28.255762 (+1571758us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54004970d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:28.255931 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685272320}
W20260709 12:19:28.510797 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72324.1us
I20260709 12:19:28.596115 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.596591 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:28.560328 26338 rpcz_store.cc:276] 0709 12:19:26.951143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.506944 (+555801us) service_pool.cc:224] Handling call
0709 12:19:28.365018 (+858074us) spinlock_profiling.cc:243] Waited 42.2 ms on lock 0x7b54008944d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:28.365218 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97099520}
I20260709 12:19:28.442968 25636 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:28.561487 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:28.442385 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 162389us
I20260709 12:19:28.598716 25786 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:28.599262 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.438870 30286 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:28.599933 25677 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:28.600464 30286 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.387113 30307 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:28.459515 25787 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:28.498720 30247 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.010s	sys 0.017s
I20260709 12:19:28.359935 30151 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:28.361771 30241 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:28.610194 25778 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:28.609333 26344 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:28.596735 25781 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:28.572176 30278 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:19:28.562064 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1980) took 1117 ms. Trace:
I20260709 12:19:28.612162 25784 rpcz_store.cc:276] 0709 12:19:27.444681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.265111 (+820430us) service_pool.cc:224] Handling call
0709 12:19:28.562002 (+296891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.603299 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.575260 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:28.606388 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:28.607532 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:28.501832 30216 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:19:28.609500 25780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 1973) took 1720 ms (client timeout 2119 ms). Trace:
I20260709 12:19:28.630684 26339 raft_consensus.cc:2368] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:28.623814 25779 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:28.561149 30288 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:28.630900 25780 rpcz_store.cc:269] 0709 12:19:26.888942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.442280 (+553338us) service_pool.cc:224] Handling call
0709 12:19:28.609340 (+1167060us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b54000ed654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:28.609451 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126114688}
I20260709 12:19:28.638792 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:28.602032 30307 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.635592 25786 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
W20260709 12:19:28.631237 30247 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.012s	sys 0.018s
I20260709 12:19:28.611445 30190 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:28.763309 25787 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:28.764173 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2487) took 3709 ms. Trace:
I20260709 12:19:28.764549 25787 rpcz_store.cc:276] 0709 12:19:25.054312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.636909 (+3582597us) service_pool.cc:224] Handling call
0709 12:19:28.763117 (+126208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.763911 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.764103 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.765826 25787 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:28.766425 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1963) took 2962 ms. Trace:
W20260709 12:19:28.767329 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 192234us
I20260709 12:19:28.814085 25787 rpcz_store.cc:276] 0709 12:19:25.804073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.765392 (+2961319us) service_pool.cc:224] Handling call
0709 12:19:28.765657 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.766234 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.766381 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.768091 26343 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:28.769462 25783 raft_consensus.cc:2410] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:19:28.577904 29799 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 12:19:28.637172 30120 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:28.630194 30255 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:28.628057 30241 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:28.635651 25785 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:28.766856 29901 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:19:28.578132 30140 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:28.719652 30087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.073s	sys 0.135s
W20260709 12:19:28.777050 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 402114us
I20260709 12:19:28.775702 25781 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:19:28.896852 30087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.073s	sys 0.136s
I20260709 12:19:28.897492 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2485) took 3917 ms. Trace:
I20260709 12:19:28.897933 25781 rpcz_store.cc:276] 0709 12:19:24.980550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.637048 (+3656498us) service_pool.cc:224] Handling call
0709 12:19:28.763684 (+126636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.897153 (+133469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.897430 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.898443 30286 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:28.901497 30140 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.835736 30241 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:28.773552 30197 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.911652 30197 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:28.719622 30307 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:28.928910 30226 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.701s	user 0.005s	sys 0.003s
I20260709 12:19:28.929608 30226 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.796976 30288 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:28.772675 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 337733us
I20260709 12:19:28.815611 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:28.767269 25786 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:28.785099 30216 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:28.961031 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1961) took 3157 ms. Trace:
I20260709 12:19:28.961680 30216 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.652194 30306 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.817299 25633 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
W20260709 12:19:28.773581 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.961470 25786 rpcz_store.cc:276] 0709 12:19:25.803607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.716251 (+2912644us) service_pool.cc:224] Handling call
0709 12:19:28.767089 (+ 50838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.960614 (+193525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.960956 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.968456 25786 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:28.969271 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2499) took 2873 ms. Trace:
I20260709 12:19:28.687342 30056 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:28.854423 30265 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:28.859965 30310 consensus_meta.cc:287] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.003s
I20260709 12:19:28.875515 30258 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.883250 30155 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.883955 30221 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:28.887287 30061 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:28.888522 29987 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:28.979290 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2049) took 1520 ms. Trace:
I20260709 12:19:28.979823 25637 rpcz_store.cc:276] 0709 12:19:27.458543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.458769 (+   226us) service_pool.cc:224] Handling call
0709 12:19:27.459160 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.543096 (+ 83936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.978977 (+1435881us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b54002acbd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:28.979219 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164187008}
I20260709 12:19:28.836373 30151 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.896252 30120 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:28.583686 30224 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:28.941679 30308 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:28.969211 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 153685us
I20260709 12:19:28.970993 30310 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:28.970692 25786 rpcz_store.cc:276] 0709 12:19:26.096295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.966285 (+2869990us) service_pool.cc:224] Handling call
0709 12:19:28.968206 (+  1921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.968979 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.969204 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:28.967482 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:28.986423 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:28.990245 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:28.834738 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.985239 29901 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:28.993073 25780 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:29.001602 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:29.002795 25710 raft_consensus.cc:2410] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:19:29.000447 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:28.996848 29799 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:19:28.997413 29901 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:28.996845 30310 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:28.987113 30151 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.037727 29970 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:29.052237 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53761.4us
W20260709 12:19:29.053503 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 441862us
W20260709 12:19:29.048448 25750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.020s	sys 0.049s
W20260709 12:19:29.055269 25750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.020s	sys 0.049s
I20260709 12:19:29.054169 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:29.055858 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 698348us
I20260709 12:19:29.056610 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:29.056758 26307 leader_election.cc:336] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.606s (SENT)
I20260709 12:19:29.058058 25786 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:29.048494 29901 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:29.053247 30128 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.000s
I20260709 12:19:29.054852 29799 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:29.037017 30243 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.058928 29970 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:29.072786 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2478) took 4169 ms. Trace:
I20260709 12:19:29.084152 25779 rpcz_store.cc:276] 0709 12:19:24.903887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.022883 (+3118996us) service_pool.cc:224] Handling call
0709 12:19:28.023255 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.652975 (+629720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.072442 (+419467us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54000ee054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:29.072706 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380518144}
I20260709 12:19:29.073683 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1074) took 2123 ms. Trace:
I20260709 12:19:29.085475 26340 rpcz_store.cc:276] 0709 12:19:26.950542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.950697 (+   155us) service_pool.cc:224] Handling call
0709 12:19:26.955948 (+  5251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:27.633484 (+677536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.073411 (+1439927us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54004975d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:29.073610 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1512827264}
I20260709 12:19:29.079402 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1952) took 4411 ms. Trace:
I20260709 12:19:29.088690 25778 rpcz_store.cc:276] 0709 12:19:24.668062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.698839 (+3030777us) service_pool.cc:224] Handling call
0709 12:19:28.610023 (+911184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.610582 (+   559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.079160 (+468578us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54000ed650. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:29.079329 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2094661376}
I20260709 12:19:29.090480 25778 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:29.091475 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 771) took 2434 ms. Trace:
I20260709 12:19:29.091919 25778 rpcz_store.cc:276] 0709 12:19:26.657489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.089828 (+2432339us) service_pool.cc:224] Handling call
0709 12:19:29.090240 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:29.091168 (+   928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.091407 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:29.078087 30151 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.051774 30120 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.083714 29970 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:29.112160 29964 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.028s	sys 0.048s
I20260709 12:19:29.132757 30243 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.134393 30308 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:29.095028 30241 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.139425 30307 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.141399 30241 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.142604 30307 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.111230 30226 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.065471 30128 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.116736 30306 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.147567 29970 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:29.159344 30337 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:19:29.150113 30297 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:29.144073 30255 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:29.172788 30297 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:29.187676 30318 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.680s	user 0.000s	sys 0.009s
I20260709 12:19:29.189347 30297 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:29.233340 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 1.02307e+06us
I20260709 12:19:28.801715 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2357) took 1335 ms. Trace:
W20260709 12:19:29.237162 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 799593us
I20260709 12:19:29.237501 25629 rpcz_store.cc:276] 0709 12:19:27.466016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.554143 (+ 88127us) service_pool.cc:224] Handling call
0709 12:19:27.574825 (+ 20682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:27.580797 (+  5972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:27.580845 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:27.581772 (+   927us) log.cc:844] Serialized 26 byte log entry
0709 12:19:28.238719 (+656947us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b180025ffc0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:19:28.480246 (+241527us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:28.480378 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.480498 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:28.801260 (+320762us) raft_consensus.cc:1685] finished
0709 12:19:28.801283 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.801626 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5604,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":47658,"spinlock_wait_cycles":5690240,"thread_start_us":241180,"threads_started":1,"wal-append.queue_time_us":1217789}
W20260709 12:19:29.242229 25630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.007s	sys 0.005s
W20260709 12:19:29.242811 25630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.007s	sys 0.005s
I20260709 12:19:29.243594 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:29.245959 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:29.246904 25631 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:29.252521 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:29.254069 25632 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:29.255352 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:29.256000 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:29.256414 25638 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.258667 25637 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:29.259210 25633 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:29.244884 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2369) took 1148 ms. Trace:
I20260709 12:19:29.266711 25630 rpcz_store.cc:276] 0709 12:19:28.096638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.096774 (+   136us) service_pool.cc:224] Handling call
0709 12:19:28.097123 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:28.354456 (+257333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.354511 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:28.386043 (+ 31532us) log.cc:844] Serialized 52 byte log entry
0709 12:19:29.243959 (+857916us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:29.244126 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.244325 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.244424 (+    99us) raft_consensus.cc:1685] finished
0709 12:19:29.244456 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.244766 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1385,"raft.queue_time_us":3325,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4183,"thread_start_us":975871,"threads_started":2,"wal-append.queue_time_us":560329}
I20260709 12:19:29.272332 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.274025 25704 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:29.274437 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:29.276511 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.285527 25638 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:29.285816 25706 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:29.277475 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.288131 25708 raft_consensus.cc:2393] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 6: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:29.289484 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:29.277520 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.291850 25707 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:29.278863 25709 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:29.296679 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.297624 25709 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:29.279682 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.300209 25712 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:29.301554 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:29.282176 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:29.304045 25631 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:29.305059 25713 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:29.275774 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:29.307761 25704 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:29.280745 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.311545 25710 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:29.280048 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.319856 25705 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:29.324997 25710 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
W20260709 12:19:29.341141 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 284407us
I20260709 12:19:29.354612 25778 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.357268 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:29.363327 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 299191us
W20260709 12:19:29.369161 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.371001 30022 raft_consensus.cc:2764] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:19:29.371986 29964 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:29.397516 25779 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.374001 30079 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:29.406301 25709 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:29.409111 30216 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.410152 30296 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:29.407359 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 136163us
I20260709 12:19:29.410893 30228 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:29.419443 25707 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:29.409888 30226 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.399639 29964 consensus_queue.cc:1059] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:19:29.411988 30241 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:29.423659 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:29.434731 30151 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:29.436632 30151 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:29.437873 30274 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:29.439433 30022 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:29.439954 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:29.440367 30310 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:29.443009 30190 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:29.450421 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2489) took 3495 ms. Trace:
I20260709 12:19:29.453418 25638 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:29.451402 25785 rpcz_store.cc:276] 0709 12:19:25.955263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.828911 (+2873648us) service_pool.cc:224] Handling call
0709 12:19:28.829372 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.829668 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:29.355773 (+526105us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.446240 (+ 90467us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:29.446350 (+   110us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:29.446469 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.446643 (+   174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.450139 (+  3496us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":223640,"prepare.queue_time_us":185475,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":387262,"threads_started":2}
I20260709 12:19:29.453874 30216 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.457621 25709 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:29.459818 30216 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:29.460639 30128 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.442279 30056 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:29.469050 30056 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.471813 30221 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.472138 30056 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.472708 29987 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:29.473171 30155 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:29.473351 29987 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:29.461612 30190 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:29.474584 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:29.475157 29987 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.474377 30155 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.476838 29987 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:29.475188 30310 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:29.479082 30310 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:29.479187 30228 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:29.480748 30310 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.480912 30228 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.481921 30275 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.468111 30297 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:29.482524 30275 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.451360 30022 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:29.475740 30265 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:29.461112 25750 leader_election.cc:400] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:29.488575 25750 leader_election.cc:403] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:19:29.487927 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:29.485150 30310 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:29.472715 30221 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.450140 29799 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:19:29.492682 30274 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:29.492759 30155 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.494674 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2488) took 3539 ms. Trace:
I20260709 12:19:29.472035 30241 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:29.495285 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 55566.9us
I20260709 12:19:29.498982 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:29.511684 30241 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.515492 30345 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:29.516517 30301 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:29.517016 30341 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:29.517907 30301 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:29.519515 30079 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:29.519917 30079 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:29.520897 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 763) took 4171 ms. Trace:
I20260709 12:19:29.515200 30352 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:29.516575 30345 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.528707 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:29.530104 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.530411 30352 consensus_queue.cc:1243] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:29.533605 29901 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.536098 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.536710 29901 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.537854 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:29.538756 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.521312 30079 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.540762 30079 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.542712 29962 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:29.543038 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:29.543479 29962 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:29.544261 26336 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:29.545060 29962 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.521476 25780 rpcz_store.cc:276] 0709 12:19:25.349654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.685587 (+3335933us) service_pool.cc:224] Handling call
0709 12:19:28.788135 (+102548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.997579 (+209444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.520347 (+522768us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54002585d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:29.520694 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470331648}
I20260709 12:19:29.547721 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:29.552855 30221 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.501399 30274 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.532907 30216 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:29.553128 30319 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.776s	user 0.003s	sys 0.000s
I20260709 12:19:29.531296 30226 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.559249 25704 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:29.560680 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2491) took 3604 ms. Trace:
I20260709 12:19:29.564105 25781 rpcz_store.cc:276] 0709 12:19:25.955883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.898753 (+2942870us) service_pool.cc:224] Handling call
0709 12:19:28.899100 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.899337 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:29.354408 (+455071us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.491124 (+136716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:29.502407 (+ 11283us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:29.531876 (+ 29469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.559012 (+ 27136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.560474 (+  1462us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154163,"prepare.queue_time_us":116069,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":82903,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":27114,"thread_start_us":389841,"threads_started":2,"wal-append.queue_time_us":111278}
I20260709 12:19:29.560811 30226 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:29.566821 30275 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:29.562359 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:29.558257 30216 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:29.502401 30296 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.583825 30216 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.585564 30296 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.569135 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 762) took 4545 ms. Trace:
I20260709 12:19:29.570586 25630 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:19:29.586254 25784 rpcz_store.cc:276] 0709 12:19:25.023700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.637129 (+3613429us) service_pool.cc:224] Handling call
0709 12:19:28.762870 (+125741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.786831 (+ 23961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.786864 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:28.786906 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:28.786936 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.568569 (+781633us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b5400258ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:29.568636 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.568888 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54545152}
I20260709 12:19:29.588918 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:29.590397 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.591647 25784 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:29.592885 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.573223 29970 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.595538 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:29.596928 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.597354 29970 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:29.597612 25785 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.604453 30330 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:29.605149 30330 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:29.607595 30330 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.475296 30190 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.623188 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1004) took 2091 ms. Trace:
I20260709 12:19:29.561792 30022 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.624033 30224 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.625136 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 770) took 3207 ms. Trace:
I20260709 12:19:29.625604 25786 rpcz_store.cc:276] 0709 12:19:26.417885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.987185 (+2569300us) service_pool.cc:224] Handling call
0709 12:19:28.991779 (+  4594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:29.058482 (+ 66703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.624542 (+566060us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b54001a72d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:29.624928 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151930752}
I20260709 12:19:29.627403 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.628484 30083 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:29.629403 30083 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.575596 25750 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:29.550315 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:29.579413 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.574872 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:29.581187 30221 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:29.567747 29964 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:29.573583 30345 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.610821 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:29.612448 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 806) took 1548 ms. Trace:
I20260709 12:19:29.633347 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.613580 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1010) took 1344 ms. Trace:
W20260709 12:19:29.614282 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.623733 25634 rpcz_store.cc:276] 0709 12:19:27.531777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.752967 (+221190us) service_pool.cc:224] Handling call
0709 12:19:28.277119 (+524152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:28.277520 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.277556 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:28.278106 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 12:19:28.763205 (+485099us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:28.763250 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.763330 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:28.916871 (+153541us) raft_consensus.cc:1685] finished
0709 12:19:28.916906 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.622681 (+705775us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54001d79d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:29.622987 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":592360,"raft.queue_time_us":77334,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":5083,"spinlock_wait_cycles":1174796160,"thread_start_us":434134,"threads_started":1,"wal-append.queue_time_us":636649}
I20260709 12:19:29.575511 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.580487 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:29.608924 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:29.502218 25787 rpcz_store.cc:276] 0709 12:19:25.954899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.818293 (+2863394us) service_pool.cc:224] Handling call
0709 12:19:28.818607 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.818804 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:29.014707 (+195903us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.493411 (+478704us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.493799 (+   388us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:29.493906 (+   107us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:29.494045 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.494191 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.494450 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133067,"prepare.queue_time_us":249242,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":98,"raft.queue_time_us":89359,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":162,"thread_start_us":341787,"threads_started":1,"wal-append.queue_time_us":167153}
I20260709 12:19:29.623492 30190 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:29.632210 25779 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:29.633747 26336 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:29.631675 25631 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:29.531119 30241 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.634596 29964 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:29.657687 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 1989) took 2100 ms. Trace:
I20260709 12:19:29.659397 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:29.658708 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 135992us
I20260709 12:19:29.649355 30087 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:29.648033 29962 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:29.659569 25778 rpcz_store.cc:276] 0709 12:19:27.557209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.092860 (+1535651us) service_pool.cc:224] Handling call
0709 12:19:29.093241 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:29.355476 (+262235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.648615 (+293139us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54001a7cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:29.657622 (+  9007us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598894464}
I20260709 12:19:29.671645 25786 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:29.640131 25635 rpcz_store.cc:276] 0709 12:19:28.269219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.278761 (+  9542us) service_pool.cc:224] Handling call
0709 12:19:28.279096 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.279301 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:28.365100 (+ 85799us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.973703 (+608603us) spinlock_profiling.cc:243] Waited 940 us on lock 0x7b18001f75c0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:29.529253 (+555550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:29.613035 (+ 83782us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:29.613111 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.613221 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.613380 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8042,"prepare.queue_time_us":85598,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":55467,"raft.queue_time_us":621605,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":2425,"spinlock_wait_cycles":2161536,"thread_start_us":457360,"threads_started":1}
I20260709 12:19:29.656805 30224 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:29.635622 26342 rpcz_store.cc:276] 0709 12:19:28.064190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.064596 (+   406us) service_pool.cc:224] Handling call
0709 12:19:28.081804 (+ 17208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.082025 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:28.097989 (+ 15964us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.803059 (+705070us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b1800419aa0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a 00007f29b5999a2a
0709 12:19:29.084364 (+281305us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.458320 (+373956us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b1800419aa0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:29.576240 (+117920us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b1800419aa0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:29.578726 (+  2486us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b1800419aa0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:29.611573 (+ 32847us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:29.611684 (+   111us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:29.611790 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.611929 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.612154 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2234,"prepare.queue_time_us":15542,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":3806,"raft.queue_time_us":545852,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":13656,"spinlock_wait_cycles":43448576,"thread_start_us":280754,"threads_started":1,"wal-append.queue_time_us":1026814}
I20260709 12:19:29.658908 29901 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.658519 30226 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.658634 30100 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:29.684333 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:29.664708 30087 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:29.658803 30237 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:29.652436 30221 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.686185 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.693933 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:29.696142 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:29.692744 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 124219us
I20260709 12:19:29.699049 25781 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:29.702734 25704 raft_consensus.cc:2368] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:29.704201 30242 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:29.704460 30310 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:29.705173 30310 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:29.698374 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 118336us
I20260709 12:19:29.706439 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:29.706573 30310 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.710794 30128 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:29.710649 30226 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.712545 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:29.685551 30241 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.700126 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.729815 29964 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.731247 30226 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:29.731614 29964 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.732901 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:29.727329 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2490) took 3681 ms. Trace:
I20260709 12:19:29.698661 30345 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.740793 30345 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.742842 30238 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:29.743563 30238 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.743871 30345 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:29.744612 30237 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.745831 30238 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:29.747205 25707 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: [7.6-10.7]
I20260709 12:19:29.747421 30237 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:29.760448 30237 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:29.761133 30237 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.761705 30237 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:29.775214 30087 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.776916 30087 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:29.779162 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:29.779994 26343 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:29.783046 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:29.787628 25638 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:29.809881 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:29.813709 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:29.816527 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:29.817551 26338 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:19:29.820996 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 64333.9us
W20260709 12:19:29.822611 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 318788us
I20260709 12:19:29.815814 25603 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:29.812496 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 221042us
I20260709 12:19:29.737488 25783 rpcz_store.cc:276] 0709 12:19:25.955569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.829989 (+2874420us) service_pool.cc:224] Handling call
0709 12:19:28.861205 (+ 31216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:28.861649 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.861689 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:28.862210 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.060612 (+198402us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:29.094115 (+ 33503us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.498841 (+404726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.529576 (+ 30735us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.616462 (+ 86886us) raft_consensus.cc:1685] finished
0709 12:19:29.616484 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.637356 (+ 20872us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276875,"prepare.queue_time_us":34815,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":345394,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":423,"thread_start_us":253455,"threads_started":2,"wal-append.queue_time_us":259688}
I20260709 12:19:29.829173 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.835549 30238 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.836302 30238 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.705097 30242 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.842077 26342 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:19:29.843775 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 114962us
I20260709 12:19:29.716054 30100 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:29.826318 30330 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:29.827461 30047 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:29.837502 30216 raft_consensus.cc:2764] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:19:29.852849 30281 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:29.867121 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:29.859002 25532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 302) took 41887 ms (client timeout 15000 ms). Trace:
I20260709 12:19:29.860306 30221 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:29.842327 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:29.863258 25782 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:29.854627 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:29.868834 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:29.869864 30100 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:29.877389 26343 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:29.878747 25781 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:29.864089 30345 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:29.827777 30241 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.914230 25783 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.882191 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.883988 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.886924 26339 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:29.886791 30083 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:29.874500 30047 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.886848 26335 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:29.893354 30242 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:29.913300 25787 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:29.913779 25603 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:29.922178 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:29.924417 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:29.930531 25785 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:29.935022 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.902577 30079 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 12:19:29.945796 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 164475us
I20260709 12:19:29.955255 25704 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:29.957319 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:29.953711 29962 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:29.926828 30345 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:29.935420 30341 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:29.964726 25782 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:19:29.960440 29962 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.970355 25713 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:19:29.975359 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:29.978775 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:29.988225 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71580us
I20260709 12:19:29.970280 29962 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.994683 30321 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:29.995379 30321 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:29.997545 30321 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:29.998035 30322 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:30.003152 30357 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:19:30.005095 30047 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:30.009104 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:30.031839 30155 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:30.032407 30155 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.032796 30155 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:30.033612 30079 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:30.034598 30275 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.034842 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.035465 30275 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.037642 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:30.038295 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72574.6us
W20260709 12:19:30.044302 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 60755.9us
I20260709 12:19:30.046271 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.045379 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.057210 25784 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:30.034257 30079 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.066884 25630 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:30.070379 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:30.072466 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:30.074844 30321 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:30.075502 30238 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:30.075871 30321 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:30.076344 30321 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:30.076825 30321 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:30.074930 25704 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:30.078938 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:30.079743 25709 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:30.080893 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:30.081612 25711 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:30.084694 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:30.085390 29987 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.002s
I20260709 12:19:30.087990 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:30.088878 29901 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:30.090708 25709 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:30.091858 30175 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:30.093736 30175 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:30.094121 30175 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:30.095096 30175 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.097044 30322 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:30.066807 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.096271 30079 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.092001 30242 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:19:30.103284 25782 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:19:30.097864 25637 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:30.097872 30175 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:30.119628 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 95757.8us
I20260709 12:19:30.061213 30241 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:30.136579 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:30.132299 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:30.147634 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 73748.1us
I20260709 12:19:30.151464 30321 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.153041 30100 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:30.153301 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:30.154110 30100 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.154357 25709 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.154649 30100 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:30.162446 30341 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.163609 30341 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.162716 30275 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:30.177162 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.183491 25786 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:30.091751 30155 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.196578 25783 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
W20260709 12:19:29.867818 25532 rpcz_store.cc:269] 0709 12:18:47.971489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.972491 (+  1002us) service_pool.cc:224] Handling call
0709 12:19:29.858708 (+41886217us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:22.658907 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:22.659648 (+   741us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:22.659696 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:19:22.659722 (+    26us) tablet.cc:662] Decoding operations
0709 12:19:22.660395 (+   673us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:22.660494 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:22.660533 (+    39us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:19:22.660935 (+   402us) tablet.cc:701] Row locks acquired
0709 12:19:22.660959 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:19:22.661149 (+   190us) write_op.cc:270] Start()
0709 12:19:22.661293 (+   144us) write_op.cc:276] Timestamp: P: 1783599562661088 usec, L: 0
0709 12:19:22.661323 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:19:22.662141 (+   818us) log.cc:844] Serialized 1307 byte log entry
0709 12:19:23.385688 (+723547us) op_driver.cc:464] REPLICATION: finished
0709 12:19:23.580089 (+194401us) write_op.cc:301] APPLY: starting
0709 12:19:23.584605 (+  4516us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:23.764739 (+180134us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:23.787564 (+ 22825us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:23.789755 (+  2191us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:23.790046 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 12:19:23.790109 (+    63us) write_op.cc:312] APPLY: finished
0709 12:19:23.790346 (+   237us) log.cc:844] Serialized 41 byte log entry
0709 12:19:23.791189 (+   843us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:23.791300 (+   111us) write_op.cc:454] Released schema lock
0709 12:19:23.791359 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:23.791787 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:23.791829 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":174822,"apply.run_cpu_time_us":9027,"apply.run_wall_time_us":228395,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16685,"num_ops":3,"prepare.queue_time_us":34421,"prepare.run_cpu_time_us":65504,"prepare.run_wall_time_us":439500,"raft.queue_time_us":211399,"raft.run_cpu_time_us":26042,"raft.run_wall_time_us":121280,"replication_time_us":719772,"thread_start_us":776038,"threads_started":4,"wal-append.queue_time_us":448771,"wal-append.run_cpu_time_us":39615,"wal-append.run_wall_time_us":1680452}]]}
I20260709 12:19:30.229957 25637 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:30.233254 25709 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:19:30.234246 25782 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:30.171970 30321 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:30.193012 30274 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.024s	sys 0.018s
I20260709 12:19:30.061436 30345 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.263824 25536 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 11, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.244729 30310 raft_consensus.cc:2764] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:19:30.274200 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44790.6us
I20260709 12:19:30.276577 30281 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:30.299000 30281 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:19:30.267127 30274 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.024s	sys 0.018s
I20260709 12:19:30.299364 30281 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:30.300361 30281 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.301730 30281 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:30.277757 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 215553us
I20260709 12:19:30.303001 25676 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:30.305836 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:30.287744 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:30.288051 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:30.267395 30275 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.306988 29901 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:30.307113 30275 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.307687 29901 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:30.308668 29901 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.288676 25711 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:30.288914 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:30.315472 30237 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.000s
I20260709 12:19:30.316005 30237 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.289754 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:30.290923 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:30.305853 25780 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:30.317297 26339 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:30.285712 30100 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.289517 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:30.265985 25536 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.339056 25783 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:30.346697 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.309305 30275 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:30.335245 30190 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.004s
I20260709 12:19:30.349995 26335 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:30.351274 30241 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:19:30.351590 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45804.6us
I20260709 12:19:30.364715 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.352289 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:30.354306 30321 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:30.367560 25633 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:30.355371 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:30.366539 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.377846 25638 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:30.379187 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:30.365846 25634 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:30.381210 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:30.381636 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:30.381255 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 92037.9us
I20260709 12:19:30.387933 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.395315 25632 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:30.367767 25536 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.368359 30356 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.401171 25634 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:30.392181 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:30.363152 30379 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.376037 29987 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.364320 30155 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:30.404094 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:30.367602 30341 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.362478 29962 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.434093 25635 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:30.465921 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 113826us
I20260709 12:19:30.505669 25785 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6.
I20260709 12:19:30.394155 30238 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:30.431421 30128 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:30.431957 29964 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.462824 30237 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:30.513136 29962 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:30.514075 25631 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:30.527405 29964 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.514051 30359 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:30.531026 30359 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:30.531441 30359 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.514127 29987 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.536156 29987 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.537691 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 780) took 1168 ms. Trace:
I20260709 12:19:30.537739 30128 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:30.538450 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44256.2us
I20260709 12:19:30.538632 25784 rpcz_store.cc:276] 0709 12:19:29.369561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.592460 (+222899us) service_pool.cc:224] Handling call
0709 12:19:30.537421 (+944961us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b540011f154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:30.537602 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883114880}
I20260709 12:19:30.539269 30128 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.540506 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:30.530193 29964 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.509939 25536 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 10, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 10 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.544119 26342 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:30.496837 30175 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.432641 30228 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:30.574198 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.575735 30297 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:30.576788 30380 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:19:30.544853 30352 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:19:30.585567 30352 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:30.545085 30341 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:30.576624 26343 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:19:30.577807 30375 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:19:30.541767 30310 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:30.582293 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:30.464133 30241 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.141s
W20260709 12:19:30.468982 30377 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
I20260709 12:19:30.588246 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 778) took 1219 ms. Trace:
I20260709 12:19:30.577186 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:30.587500 25536 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 8, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 8 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.590191 26338 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:30.596312 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:30.580925 30228 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.597323 25787 rpcz_store.cc:276] 0709 12:19:29.368784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.010373 (+641589us) service_pool.cc:224] Handling call
0709 12:19:30.042454 (+ 32081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.123003 (+ 80549us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:30.149674 (+ 26671us) log.cc:844] Serialized 14 byte log entry
0709 12:19:30.582149 (+432475us) log.cc:844] Serialized 14 byte log entry
0709 12:19:30.587873 (+  5724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:30.587930 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:30.587980 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.588057 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.588183 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62289,"prepare.queue_time_us":124837,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":103,"raft.queue_time_us":22671,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":4831,"thread_start_us":354657,"threads_started":2,"wal-append.queue_time_us":169149}
I20260709 12:19:30.463914 30356 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.607136 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:30.506896 30379 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.606030 30228 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:30.645298 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57488.7us
I20260709 12:19:30.648278 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 773) took 1282 ms. Trace:
I20260709 12:19:30.648761 25781 rpcz_store.cc:276] 0709 12:19:29.366464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.941298 (+574834us) service_pool.cc:224] Handling call
0709 12:19:29.967340 (+ 26042us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:29.989071 (+ 21731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:29.989114 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:30.054786 (+ 65672us) log.cc:844] Serialized 52 byte log entry
0709 12:19:30.247045 (+192259us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:30.324103 (+ 77058us) log.cc:844] Serialized 14 byte log entry
0709 12:19:30.367978 (+ 43875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.368057 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:30.647981 (+279924us) raft_consensus.cc:1685] finished
0709 12:19:30.648005 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.648208 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47051,"prepare.queue_time_us":81840,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":2414,"raft.queue_time_us":49273,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":27471,"thread_start_us":116006,"threads_started":1,"wal-append.queue_time_us":241450}
I20260709 12:19:30.587441 30297 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.594673 30190 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:30.631949 30321 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:30.655061 30100 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:30.586830 30352 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.628737 30310 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.640748 25536 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 9, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 9 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.596158 30175 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.641538 29964 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:19:30.666568 29964 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:30.666942 29964 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.667151 30356 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.667347 29964 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:30.667776 30356 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.668359 30356 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.666553 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:30.659826 30321 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:30.676990 30274 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:30.672544 30356 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:30.673266 30310 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.681974 30321 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:30.696571 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:30.689472 25536 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:30.715416 30175 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:30.720389 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2010) took 1182 ms. Trace:
I20260709 12:19:30.721076 25779 rpcz_store.cc:276] 0709 12:19:29.520338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.653333 (+132995us) service_pool.cc:224] Handling call
0709 12:19:30.696469 (+1043136us) spinlock_profiling.cc:243] Waited 969 us on lock 0x7b5c00237e0c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b92f9161 00007f29b949aa79 00007f29ba9549b6 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:30.702502 (+  6033us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2228608}
I20260709 12:19:30.726025 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:30.727501 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:30.728396 25635 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:30.730763 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.731758 25752 leader_election.cc:400] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
W20260709 12:19:30.730469 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 298525us
I20260709 12:19:30.742060 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:30.726655 30274 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.008s
W20260709 12:19:30.760496 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 188623us
I20260709 12:19:30.761454 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:30.731074 25536 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 9, leader changed from <none> to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 9 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.772488 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:30.774211 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.782294 30379 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:30.782908 30379 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:30.784214 30379 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.784755 30379 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.785784 30228 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.787673 30310 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:30.798599 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 785) took 1336 ms. Trace:
I20260709 12:19:30.800917 25710 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:30.811529 25785 rpcz_store.cc:276] 0709 12:19:29.463098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.935694 (+472596us) service_pool.cc:224] Handling call
0709 12:19:30.798361 (+862667us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54000eef54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:30.798513 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976862592}
I20260709 12:19:30.803212 30388 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:19:30.803781 30374 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
I20260709 12:19:30.805022 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2600) took 1022 ms. Trace:
I20260709 12:19:30.765837 29901 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.814985 25713 rpcz_store.cc:276] 0709 12:19:29.787268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.812857 (+ 25589us) service_pool.cc:224] Handling call
0709 12:19:29.847463 (+ 34606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.977771 (+130308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.804665 (+826894us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54002ec2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:30.804948 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280869760}
W20260709 12:19:30.806847 25780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 782) took 1436 ms (client timeout 1777 ms). Trace:
W20260709 12:19:30.817934 25780 rpcz_store.cc:269] 0709 12:19:29.370467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.549470 (+179003us) service_pool.cc:224] Handling call
0709 12:19:30.806671 (+1257201us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b5400380254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:30.806786 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532373760}
I20260709 12:19:30.759490 30359 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:30.819986 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 415974us
I20260709 12:19:30.812651 25636 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:30.771142 30241 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:30.815624 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 83436us
W20260709 12:19:30.826333 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 219320us
I20260709 12:19:30.842936 29901 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.843901 25536 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.820935 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:30.820719 30330 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:19:30.818714 30310 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:19:30.847344 30330 consensus_queue.cc:1243] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:30.847903 30330 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:30.848279 30330 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:30.831502 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:30.847028 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:30.846138 25536 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.854386 25635 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:30.815382 30297 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.857306 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:30.863240 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 90831us
I20260709 12:19:30.864189 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 791) took 1396 ms. Trace:
I20260709 12:19:30.856153 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:30.846282 29901 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:30.867251 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.822988 30241 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.871675 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:30.864941 25786 rpcz_store.cc:276] 0709 12:19:29.468048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.231728 (+763680us) service_pool.cc:224] Handling call
0709 12:19:30.232509 (+   781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.322788 (+ 90279us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:30.685138 (+362350us) log.cc:844] Serialized 14 byte log entry
0709 12:19:30.863827 (+178689us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:30.863893 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:30.863937 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.864005 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.864130 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10416,"prepare.queue_time_us":46098,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":36,"thread_start_us":448754,"threads_started":2}
I20260709 12:19:30.849740 30385 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:30.856376 30237 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:30.869293 26343 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:30.863848 30310 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.863690 30345 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:30.885630 30255 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:30.888212 30322 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:19:30.888931 30301 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:30.908185 30301 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:30.908555 30301 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:30.909435 30301 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.910651 30255 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:30.910810 30395 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:19:30.911345 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.912020 25635 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:30.911702 30255 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.913861 30190 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.914531 30190 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:30.916874 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.917675 30352 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:19:30.919571 30301 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:30.894587 30128 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:30.923231 30128 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:30.924644 30128 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.916790 30190 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:30.876447 30241 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.890133 25536 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 8, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 8 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.876302 30379 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:30.892748 25521 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0ae979356d694a09be4b0cd976ea354a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:30.893832 30359 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.892781 30297 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:30.849164 30330 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.942065 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.874369 30390 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:30.898756 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 77858.4us
I20260709 12:19:30.900607 30351 raft_consensus.cc:2764] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:19:30.901466 30389 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:30.901726 30237 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:30.950440 25536 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 6, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 6 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.902357 30022 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.904098 29987 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:30.906107 29962 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:30.880358 30385 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.938921 30083 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:30.933602 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:30.957605 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:30.936085 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64480.7us
I20260709 12:19:30.940508 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:30.942804 25635 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:30.945250 25632 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:30.935794 30388 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.960960 25704 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:30.929639 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.962637 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:30.964249 25636 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:30.965623 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:30.954013 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 66257.2us
I20260709 12:19:30.956139 25707 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:30.968290 25711 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:30.968626 25751 leader_election.cc:336] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.826s (SENT)
I20260709 12:19:30.969132 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:30.931612 30310 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.971045 25710 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:30.971089 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:30.975186 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:30.975767 25630 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.970016 25536 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 8, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 8 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:30.954845 29962 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:30.988016 29962 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.989084 29962 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.954996 29987 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:30.994684 30390 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:30.995184 29987 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.996248 29987 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.997243 30390 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:30.956166 30385 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:30.997566 29964 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.998067 30238 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:30.998742 30238 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.000831 30238 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:31.001662 30356 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:31.002156 30356 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:31.003193 30356 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.976184 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:30.979768 25536 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 8, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 8 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:30.981161 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:30.981580 30321 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.964078 30228 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.984215 30310 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:30.985401 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:30.985960 30330 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:30.986627 30083 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:30.970968 30389 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:19:30.974701 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.005839 25704 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:30.963433 30255 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:31.006620 30367 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 1.021s	user 0.007s	sys 0.000s
I20260709 12:19:31.021785 30237 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.004530 30356 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:31.029011 30390 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:31.074678 30190 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.075296 30190 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.076140 30322 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:31.076676 30322 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:31.077502 30190 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:31.078717 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:31.029698 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.079787 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:31.080358 26338 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:31.030356 30241 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.035465 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.039969 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:31.040558 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.040671 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.041496 30255 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:31.041687 30391 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:19:31.086449 26335 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:31.042285 25536 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 8, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 8 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:31.043462 30083 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:31.046917 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 85607.8us
I20260709 12:19:31.047540 25631 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:31.048652 30321 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:31.039969 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:31.052132 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.053592 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.055522 30389 consensus_queue.cc:1243] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:31.090574 30255 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.055833 29964 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:31.056070 30379 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:19:31.057497 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 92522.5us
I20260709 12:19:31.059397 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:30.971411 30388 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.067775 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 932) took 1653 ms. Trace:
I20260709 12:19:31.069147 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:31.013702 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.070709 30228 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:31.071431 30385 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.086418 26343 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:31.086681 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 109757us
I20260709 12:19:31.059592 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.099731 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:31.100333 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.107615 26335 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:19:31.107764 25704 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:31.109349 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.110409 25704 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:31.111368 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.112167 25704 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:31.097608 25536 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 6, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 6 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:31.113018 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.113891 25704 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:31.114208 25536 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 8, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 8 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:31.115615 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.116575 25704 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:31.116466 25536 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 5, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 5 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:31.117682 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.100497 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.118459 25704 raft_consensus.cc:2376] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11. Re-sending same reply.
I20260709 12:19:31.119561 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.120679 25704 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:31.121661 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.122470 25708 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:31.127180 25636 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:31.122826 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.128451 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.129473 25713 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:31.102854 26341 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:31.077792 30322 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.130695 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:31.131269 30322 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.131278 25633 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:31.124928 30385 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:31.138650 30379 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:31.139129 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.139599 30379 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.140278 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.141054 25710 raft_consensus.cc:2410] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:19:31.142094 25710 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:31.144863 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.144465 25536 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 7, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 7 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:31.134158 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.146241 25707 raft_consensus.cc:2376] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10. Re-sending same reply.
I20260709 12:19:31.147063 26335 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
W20260709 12:19:31.142949 30341 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.012s	sys 0.026s
I20260709 12:19:31.151925 25710 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 5. Current term is 6. Ops: []
I20260709 12:19:31.059744 30238 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:19:31.154580 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.155153 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.157624 25709 raft_consensus.cc:2368] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:19:31.096760 30297 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.168263 25711 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:31.163806 25706 raft_consensus.cc:2368] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12.
I20260709 12:19:31.164915 25707 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:31.171512 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.162678 25705 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
W20260709 12:19:31.154165 30341 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.012s	sys 0.026s
I20260709 12:19:31.153349 29901 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:31.185611 29901 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.186015 29901 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:31.186084 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.151670 30128 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:19:31.186664 30400 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.186717 25631 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:31.187489 30400 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.188318 25632 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:19:31.188699 25632 pending_rounds.cc:85] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 7
I20260709 12:19:31.188902 25632 pending_rounds.cc:107] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:19:31.189157 25632 raft_consensus.cc:2889] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:31.189765 30400 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:31.180109 30297 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.192937 30297 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:31.156498 30228 consensus_queue.cc:1243] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:31.195025 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:31.195449 30255 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.196280 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:31.197954 30255 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:31.201021 29987 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:31.201850 29987 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:31.203325 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:31.177129 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 117831us
I20260709 12:19:31.206796 26342 raft_consensus.cc:2368] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:19:31.207139 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:31.207779 25633 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:31.209638 25630 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:31.177690 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.178303 30238 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:31.178416 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.214530 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.215245 25780 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:31.215452 25779 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:31.179343 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.104923 26340 rpcz_store.cc:276] 0709 12:19:29.414338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.432128 (+ 17790us) service_pool.cc:224] Handling call
0709 12:19:29.473200 (+ 41072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.673790 (+200590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:29.673838 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.750796 (+ 76958us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.931966 (+181170us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:29.932010 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.932096 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:30.935783 (+1003687us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:31.067343 (+131560us) raft_consensus.cc:1685] finished
0709 12:19:31.067366 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:31.067565 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239287,"raft.queue_time_us":192712,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":22071,"spinlock_wait_cycles":7645440,"thread_start_us":73480,"threads_started":1,"wal-append.queue_time_us":111749}
I20260709 12:19:31.172075 25712 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:31.182039 25784 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:31.217077 25784 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:31.168293 26338 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:31.182616 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.184590 25786 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:31.217898 25787 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:31.185174 30388 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:31.190817 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.213052 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:31.197412 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:31.178951 30100 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:31.182214 30352 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:31.220068 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:31.227043 25710 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:31.227525 25709 raft_consensus.cc:2368] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:19:31.220376 25630 pending_rounds.cc:85] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 7
I20260709 12:19:31.231419 25630 pending_rounds.cc:107] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:19:31.231681 25630 raft_consensus.cc:2889] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:31.162358 30301 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:19:31.180020 30341 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:31.259625 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.260352 25780 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:31.260449 30341 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.261186 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.261061 30238 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:31.237284 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.253067 30388 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:31.263762 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.264261 30321 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:31.265175 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.265791 25778 raft_consensus.cc:2376] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:31.265937 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.266866 30351 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:31.266731 30238 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:31.267362 30351 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:31.268193 30351 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.266083 30321 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:31.269520 30351 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:31.269692 30321 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:31.228319 30100 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:31.271023 30321 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.271817 30100 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.273243 30100 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:31.238643 30241 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:31.239112 30356 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:31.240635 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.302930 25783 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:31.303112 26335 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:31.219115 30281 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:31.309543 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.242606 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.310253 25779 raft_consensus.cc:2376] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10. Re-sending same reply.
I20260709 12:19:31.242827 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.311344 25785 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.247763 29901 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.251943 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.313010 25782 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:31.232659 30330 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 12:19:31.257639 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.310623 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 74011.8us
W20260709 12:19:31.303825 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 106469us
I20260709 12:19:31.314606 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.259327 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.315313 25709 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:31.277292 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.261204 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.280570 25786 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:31.316967 25780 raft_consensus.cc:2376] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9. Re-sending same reply.
I20260709 12:19:31.286970 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:31.287464 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 56877.8us
I20260709 12:19:31.318421 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.304733 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.321220 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.321650 25778 raft_consensus.cc:2376] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11. Re-sending same reply.
I20260709 12:19:31.322373 25712 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:31.323778 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:31.324417 30022 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:31.324692 30389 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.325001 30022 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.325532 30389 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.326270 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:31.327143 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.327735 30330 consensus_queue.cc:1243] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:31.327276 26343 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:31.273255 30321 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:31.330803 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:31.331079 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.331420 25629 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.331655 25634 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:31.331357 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.331929 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:31.333678 25637 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:31.334692 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.335390 25786 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:31.327935 26344 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:31.340425 25710 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:31.345137 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.346071 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.348532 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.349699 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.351253 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:31.351934 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.351287 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.353232 25713 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:31.354823 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.355520 25780 raft_consensus.cc:2376] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14. Re-sending same reply.
I20260709 12:19:31.357985 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.358821 25636 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:31.347239 25677 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.328066 30390 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:31.362334 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.362761 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.363396 25778 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:31.322259 30228 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:31.359059 26337 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:31.363142 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.360759 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.365387 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.365749 25636 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:31.368081 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:31.321326 30356 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:31.373021 25631 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:31.373720 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:31.374922 25784 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.378224 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.309255 30255 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:19:31.379331 25783 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:31.377678 25785 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:31.375541 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:31.386581 25784 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.387003 25633 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:31.375005 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.387647 25781 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 6.
I20260709 12:19:31.351418 30330 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:31.383134 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.385668 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.388964 25632 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:31.390000 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:31.388038 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:31.378846 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 63599.8us
I20260709 12:19:31.339877 30388 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:19:31.342226 30405 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:31.369964 30128 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:31.391155 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.400278 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:31.381925 26342 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:31.398249 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.400537 25786 raft_consensus.cc:2376] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 6. Re-sending same reply.
I20260709 12:19:31.399425 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.370167 30228 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.400512 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:31.402866 25634 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:31.404093 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:31.404441 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:31.405125 26337 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:31.377781 30310 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:31.406543 25780 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:31.409190 25637 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:31.379897 30389 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:31.412364 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.412597 25713 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:19:31.413611 25632 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:31.390226 30330 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:31.419700 25784 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:19:31.422369 25783 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:19:31.413215 25603 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.416664 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.425562 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:31.426297 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.426433 25713 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:31.429333 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.393743 25787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:31.437458 25787 raft_consensus.cc:606] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:31.437855 25787 raft_consensus.cc:614] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:31.405853 30228 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:31.391501 30390 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:19:31.390964 30359 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:31.408041 30333 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:31.420192 25784 pending_rounds.cc:85] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 8
I20260709 12:19:31.478358 25784 pending_rounds.cc:107] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:19:31.478679 25784 raft_consensus.cc:2889] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:31.406379 29962 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:19:31.443835 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.461844 25713 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:31.410931 30398 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:31.476028 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.421289 30322 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:31.423746 30237 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:31.464064 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.553365 25636 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:31.406637 30379 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:19:31.428665 30281 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:19:31.428761 30301 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.542748 25780 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:31.436920 30274 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:31.429039 30389 raft_consensus.cc:2764] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:19:31.540729 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 152766us
I20260709 12:19:31.421468 30310 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:31.463073 30352 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:31.549836 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.617336 25713 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:31.544588 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.618338 25784 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:31.463240 30128 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:31.554085 25638 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:31.551672 25710 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 6.
I20260709 12:19:31.439729 30255 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
W20260709 12:19:31.573731 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 182418us
W20260709 12:19:31.573040 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 163771us
W20260709 12:19:31.545122 30414 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.217s	user 0.000s	sys 0.004s
I20260709 12:19:31.633909 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:31.419121 30405 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.592311 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:31.547722 30393 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:19:31.414826 30345 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 12:19:31.609367 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.640265 25787 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
W20260709 12:19:31.622973 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 218584us
I20260709 12:19:31.552696 30322 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.559664 30237 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.631191 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:31.594630 30398 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:31.646976 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.632416 25780 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:31.634982 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.648624 25711 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:31.636077 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.649650 25710 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:31.620677 30352 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.657722 25784 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:31.658196 30379 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.658792 30379 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.541272 30356 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.638424 29901 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:31.619485 30301 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.666352 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.674582 25780 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:31.655942 30237 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:31.662338 30352 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:31.692011 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.645561 30322 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:31.675839 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:31.710032 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:31.667601 30333 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:31.714823 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 83725.3us
I20260709 12:19:31.717537 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:31.713955 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.716717 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:31.698413 30396 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:31.696877 30389 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:31.707434 29962 consensus_queue.cc:1243] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:31.736006 25629 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:31.723147 30333 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.756341 30281 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:31.797142 25778 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:31.795053 25787 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:31.802884 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:31.762461 30274 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:31.808005 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.728360 30128 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.781942 30390 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:31.800860 30412 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:19:31.727813 30310 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.818343 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:31.813175 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.822100 30405 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:31.822381 25778 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:31.765249 30389 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:31.824788 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 195337us
I20260709 12:19:31.825752 30389 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:31.826743 26307 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:31.827414 30389 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:31.828329 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:31.829631 30416 raft_consensus.cc:2764] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:19:31.831925 30416 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:31.832616 30416 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:31.833151 25778 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:31.830576 30389 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.833796 30416 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.834643 25778 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:19:31.836987 30389 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:31.793579 30429 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:19:31.846544 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 132690us
I20260709 12:19:31.830029 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:31.874233 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44305.8us
I20260709 12:19:31.875204 26307 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:31.876385 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:31.877490 26307 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:31.879051 30416 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:31.879878 30416 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:31.881310 30416 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:31.814751 30411 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
W20260709 12:19:31.871214 26309 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.559s (SENT)
I20260709 12:19:31.884167 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:31.882957 30425 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:19:31.891211 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 174587us
I20260709 12:19:31.893972 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.813102 30390 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.812067 30418 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.864521 25634 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:31.903057 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:31.931277 25676 leader_election.cc:400] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:19:31.932571 25676 leader_election.cc:403] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:19:31.940656 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.948446 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.950271 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:31.903007 30390 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:31.952805 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.953528 25778 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:31.781414 30398 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.969884 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:31.970671 25778 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
W20260709 12:19:31.983721 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 141689us
W20260709 12:19:31.984434 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 105910us
I20260709 12:19:31.960260 30429 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.957557 30301 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:31.934862 30405 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:19:31.938879 30228 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:31.934281 30389 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:19:31.949666 30190 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:19:31.958837 30425 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:32.039726 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54908.5us
I20260709 12:19:32.017604 30421 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:31.933845 30418 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:31.960659 30274 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.038862 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:31.960251 30281 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.054368 25637 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:32.040176 30400 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:32.042966 25707 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:32.116410 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 78132.2us
I20260709 12:19:32.121281 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:32.055110 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:32.089311 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 458410us
I20260709 12:19:32.124464 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:32.126338 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:32.129889 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 258715us
W20260709 12:19:32.047076 30424 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 12:19:32.047827 30434 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:19:32.047910 30413 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
W20260709 12:19:32.047070 30422 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 12:19:32.171452 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:32.052328 30443 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:32.047159 30440 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:32.048009 30432 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:32.041937 30430 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:32.120230 30400 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:32.134166 30022 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:32.134769 30333 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:32.192245 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60579.8us
W20260709 12:19:32.224613 26309 leader_election.cc:336] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.675s (SENT)
W20260709 12:19:32.225579 26309 leader_election.cc:336] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.674s (SENT)
I20260709 12:19:32.225945 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:32.047863 30330 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.227288 30330 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.227584 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:32.228890 26309 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.917s (SENT) [suppressed 30 similar messages]
I20260709 12:19:32.042590 30345 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 12:19:32.046180 30428 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.264s
I20260709 12:19:32.232154 30428 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:32.232503 30428 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:32.233409 30428 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:32.237362 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.237869 30321 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.871s	user 0.006s	sys 0.005s
I20260709 12:19:32.238426 25636 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:32.241864 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:32.243628 30418 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:32.248339 30238 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:32.248783 30238 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:32.249643 30238 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.184409 29962 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:32.252528 30083 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:32.252934 30083 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:32.255689 25521 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0ae979356d694a09be4b0cd976ea354a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:32.257079 30432 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:32.257925 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 794) took 1035 ms. Trace:
I20260709 12:19:32.258957 30442 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.259537 30442 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.258414 25779 rpcz_store.cc:276] 0709 12:19:31.222126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.321364 (+ 99238us) service_pool.cc:224] Handling call
0709 12:19:32.257725 (+936361us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b54001243d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:32.257862 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1925340928}
I20260709 12:19:32.097579 30425 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.264937 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 801) took 1040 ms. Trace:
I20260709 12:19:32.265378 25781 rpcz_store.cc:276] 0709 12:19:31.224451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.388325 (+163874us) service_pool.cc:224] Handling call
0709 12:19:31.415500 (+ 27175us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:31.646944 (+231444us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:31.696705 (+ 49761us) log.cc:844] Serialized 14 byte log entry
0709 12:19:31.838452 (+141747us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:31.838978 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:19:31.839121 (+   143us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:32.130611 (+291490us) log.cc:844] Serialized 14 byte log entry
0709 12:19:32.192511 (+ 61900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.192613 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:32.264742 (+ 72129us) raft_consensus.cc:1685] finished
0709 12:19:32.264761 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.264883 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83954,"prepare.queue_time_us":159173,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":135,"raft.queue_time_us":287585,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":10392,"thread_start_us":564593,"threads_started":4,"wal-append.queue_time_us":183302}
I20260709 12:19:32.198670 30430 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.198179 30433 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:32.185923 30443 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.214425 30420 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:32.214960 30426 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:32.215986 30359 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:32.216189 30427 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.216681 30435 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:32.217064 30431 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:32.217532 30417 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:32.219040 30379 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:32.222810 30438 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.223309 30228 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:32.173481 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 118464us
I20260709 12:19:32.315043 30433 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:32.316258 30430 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:32.279090 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:32.297785 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:32.296113 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 805) took 1059 ms. Trace:
I20260709 12:19:32.367277 30426 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:32.267216 30083 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:32.326972 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53829.8us
W20260709 12:19:32.359355 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.363263 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:32.321162 30434 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:32.386879 26340 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:32.321144 30420 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.321455 30238 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:32.288159 30442 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:32.325197 30428 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:32.364104 30322 consensus_meta.cc:287] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.003s
I20260709 12:19:32.276630 25637 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:32.312752 30443 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:32.323959 30359 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:32.377300 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:32.386734 25635 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.367416 30417 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.395473 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2095) took 1176 ms. Trace:
I20260709 12:19:32.500581 25630 rpcz_store.cc:276] 0709 12:19:31.219542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.219761 (+   219us) service_pool.cc:224] Handling call
0709 12:19:31.220142 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:31.311387 (+ 91245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:31.311435 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:31.312159 (+   724us) log.cc:844] Serialized 52 byte log entry
0709 12:19:31.674011 (+361852us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:31.674051 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:31.674118 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:32.395157 (+721039us) raft_consensus.cc:1685] finished
0709 12:19:32.395182 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.395374 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58479,"raft.queue_time_us":57925,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":15219,"thread_start_us":319525,"threads_started":1,"wal-append.queue_time_us":369502}
W20260709 12:19:32.501672 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.383358 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:32.405865 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.385840 30433 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.368405 30431 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.376816 30389 consensus_queue.cc:1243] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:32.490775 25709 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:32.506263 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 123010us
I20260709 12:19:32.414975 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:19:32.388201 30444 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:19:32.387315 30448 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:19:32.312424 30390 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:32.512662 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 391446us
I20260709 12:19:32.498772 25705 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:32.516568 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:32.507265 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:32.487037 30420 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:32.514402 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.001s	sys 0.027s
W20260709 12:19:32.522069 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.002s	sys 0.027s
W20260709 12:19:32.522464 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 572350us
I20260709 12:19:32.524237 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:32.504714 30417 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.369458 30427 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:32.516479 30351 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.246s	user 0.015s	sys 0.001s
I20260709 12:19:32.567905 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.397475 30438 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.528494 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:32.387035 25782 rpcz_store.cc:276] 0709 12:19:31.236922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.401776 (+164854us) service_pool.cc:224] Handling call
0709 12:19:31.415830 (+ 14054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:31.692973 (+277143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:31.693009 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:31.693060 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:31.693090 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.295744 (+602654us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400258ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:32.295815 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.296028 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570077056}
W20260709 12:19:32.458313 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.584770 25753 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.006s	sys 0.028s
I20260709 12:19:32.500185 30322 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:32.459138 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.563891 25709 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:19:32.570832 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.597472 25712 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:32.597575 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.573087 25635 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:32.560467 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 807) took 1234 ms. Trace:
I20260709 12:19:32.566697 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:32.412652 30400 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:32.601189 25753 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.006s	sys 0.028s
I20260709 12:19:32.576426 30388 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:19:32.601913 25782 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:32.604146 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:32.536027 30275 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:32.605226 30275 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.605626 30275 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:32.606782 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:32.599520 25780 rpcz_store.cc:276] 0709 12:19:31.325665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.694103 (+368438us) service_pool.cc:224] Handling call
0709 12:19:31.709754 (+ 15651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:31.805434 (+ 95680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:31.805473 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:31.816526 (+ 11053us) log.cc:844] Serialized 26 byte log entry
0709 12:19:32.032287 (+215761us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:32.081907 (+ 49620us) log.cc:844] Serialized 14 byte log entry
0709 12:19:32.174569 (+ 92662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.174677 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:32.560180 (+385503us) raft_consensus.cc:1685] finished
0709 12:19:32.560208 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.560389 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52454,"prepare.queue_time_us":48762,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3835,"raft.queue_time_us":175664,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":6144,"thread_start_us":243440,"threads_started":3,"wal-append.queue_time_us":184942}
W20260709 12:19:32.612440 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.512737 30431 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:32.521979 30175 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:19:32.389539 30237 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
I20260709 12:19:32.624367 30237 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.518296 30395 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.626853 30395 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.629117 30395 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:32.565992 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 151074us
I20260709 12:19:32.630028 26309 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:32.631605 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:32.633594 26309 leader_election.cc:400] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:19:32.585604 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.635125 25708 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:32.635983 30425 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:32.636279 30442 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:32.635934 30432 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:32.395087 30083 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.637494 30175 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:32.587908 30423 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:32.638688 30423 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.639042 30423 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:32.638537 30175 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.565310 30390 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:32.533977 30301 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:32.641299 30301 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:32.641436 30458 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:32.642263 30301 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.588495 30241 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:32.586798 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58409.7us
I20260709 12:19:32.663937 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:32.387466 30435 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:32.598325 25704 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:32.623210 30430 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.636655 30442 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:32.639057 30429 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.612s	user 0.004s	sys 0.007s
I20260709 12:19:32.643397 30425 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:19:32.644335 30352 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.929s	user 0.011s	sys 0.001s
I20260709 12:19:32.634313 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:32.601939 30388 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.588058 30351 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:32.648699 30464 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:32.532240 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.678020 30400 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.679024 30442 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:32.679548 30237 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:32.677767 25638 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:32.679901 25631 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:32.679775 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:32.601191 30100 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.328s	user 0.016s	sys 0.001s
W20260709 12:19:32.692385 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58125us
I20260709 12:19:32.697467 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:32.390540 30426 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.716463 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:32.674099 30241 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:32.582779 25710 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:32.695261 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:32.719265 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:32.678975 30395 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:32.705752 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 412743us
I20260709 12:19:32.690282 30429 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:19:32.731052 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:32.705061 30341 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.731766 25778 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:32.706522 30351 consensus_queue.cc:1243] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:32.691931 30352 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.713677 25780 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:32.730548 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 404352us
I20260709 12:19:32.714697 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:32.693849 30388 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:32.690716 30435 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:32.715806 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:32.694258 30432 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:32.721612 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:32.721880 26342 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:32.722113 25782 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:32.726859 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:32.728225 25751 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.547s (SENT)
I20260709 12:19:32.651187 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:32.746577 25636 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:32.746800 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:32.747076 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 95973us
I20260709 12:19:32.731125 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:32.734800 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:32.747520 25782 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:32.739526 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:32.748000 25713 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.747973 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:32.712965 30415 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.742470 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:32.724023 30297 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:32.724053 30241 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:32.758308 30415 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.761577 30415 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:32.762096 30462 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:32.762550 25781 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:32.701172 30464 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.764662 30430 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:32.765646 30464 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:32.767529 30423 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.773141 30433 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:32.773991 25738 heartbeater.cc:646] Failed to heartbeat to 127.23.217.126:36567 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.23.217.126:36567 timed out after 15.000s (SENT)
I20260709 12:19:32.754532 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:32.713485 26309 leader_election.cc:400] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:19:32.749843 25779 raft_consensus.cc:2376] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14. Re-sending same reply.
I20260709 12:19:32.733832 30321 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:32.720877 30128 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:32.754472 30460 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:19:32.755228 30466 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.756443 30455 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:32.756906 30275 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.778657 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:32.778080 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:32.847905 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69934.6us
I20260709 12:19:32.839597 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:32.848873 25787 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.746596 30389 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.849500 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:32.777057 30429 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:32.751803 30390 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:32.777576 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64224.4us
I20260709 12:19:32.741070 30341 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.843240 30462 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:32.845013 30465 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 12:19:32.850534 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 122316us
W20260709 12:19:32.778705 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 49422.6us
I20260709 12:19:32.856758 30438 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:32.857833 30472 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
I20260709 12:19:32.860584 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:32.859515 25704 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:32.837003 30464 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:32.848284 25778 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:32.953694 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:32.865366 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 138615us
I20260709 12:19:32.923581 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:32.846565 30415 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:32.928921 25710 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:32.934893 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.860113 30228 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:32.837113 30430 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:19:32.944657 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:32.943636 25636 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:32.952250 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.867455 30390 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:32.937289 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.885360 29987 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.871634 30398 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.868708 30419 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:32.879312 30275 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:32.863271 30389 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.941759 30463 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.959733 30464 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.994957 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:32.951766 30460 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.889955 30442 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:32.867627 30417 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:32.847939 30466 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:33.010057 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.987388 30438 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.986253 30390 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:32.989795 30228 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:32.991647 30352 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.007184 30400 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.860941 30455 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:33.024996 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 249289us
I20260709 12:19:33.018194 30464 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:33.046182 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:33.003823 30421 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:19:33.038332 26339 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:33.039059 25711 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:19:33.039940 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45130.7us
W20260709 12:19:33.043982 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.046973 25710 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.007953 30398 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.058936 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:33.059772 25706 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:33.060035 30398 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.047384 30400 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.065672 30400 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.049458 30419 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:33.052640 30433 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.067890 30356 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:33.068403 30356 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:33.048672 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:33.069814 25709 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:33.069352 30356 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.029968 30405 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:33.044389 30341 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.041460 30473 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:33.045197 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:32.890290 30435 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.004602 29987 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.074286 25787 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
W20260709 12:19:33.025585 30396 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.004s	sys 0.031s
I20260709 12:19:33.083978 25713 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:33.082640 30405 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:33.088209 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.088390 30352 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:33.035185 30128 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:33.017983 30463 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.024338 30460 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.046317 30481 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.067802 30466 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.100266 25710 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
W20260709 12:19:33.085520 30330 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.858s	user 0.006s	sys 0.006s
I20260709 12:19:33.035521 30420 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.089232 30356 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:33.035944 30438 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:33.108595 30396 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.006s	sys 0.031s
W20260709 12:19:33.147914 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 203323us
I20260709 12:19:33.152654 25712 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:33.153294 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.095172 30405 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:33.154096 25783 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 12:19:33.143298 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 88141.1us
I20260709 12:19:33.157353 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:33.155316 25783 pending_rounds.cc:85] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 6
I20260709 12:19:33.162395 25783 pending_rounds.cc:107] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:19:33.162719 25783 raft_consensus.cc:2889] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:33.177430 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.144635 30455 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:33.155689 30155 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.009s	sys 0.013s
I20260709 12:19:33.028462 30462 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.191136 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:33.123565 30352 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:33.156167 30431 consensus_meta.cc:287] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.002s
I20260709 12:19:33.165391 25637 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:33.118685 30128 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.184264 29964 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:33.195470 30155 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.009s	sys 0.022s
I20260709 12:19:33.093513 30473 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.162390 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:33.161859 30275 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:19:33.163360 30345 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:33.221768 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59457.5us
W20260709 12:19:33.221832 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 471708us
I20260709 12:19:33.135558 30481 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.195683 30488 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:33.226742 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.184624 30237 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:19:33.201809 30419 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:33.232033 25706 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:33.228631 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.234455 30463 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:33.236639 30443 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.724s	user 0.009s	sys 0.002s
I20260709 12:19:33.237051 30443 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.238170 26337 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:33.239367 30460 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.228039 30492 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:33.239856 30359 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.240553 30433 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:33.241360 30359 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:33.242885 25712 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:33.243829 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:33.245019 30443 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.151945 30228 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.251767 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:33.250080 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.239580 30420 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:33.246589 30428 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.747s	user 0.013s	sys 0.000s
I20260709 12:19:33.257620 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.260972 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:33.261884 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 104626us
I20260709 12:19:33.231719 30488 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.209676 29964 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.269869 30352 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.205826 30431 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:33.271705 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 125995us
I20260709 12:19:33.276093 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:33.257719 30455 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:33.277034 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.261572 30465 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:33.279194 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.258409 30237 consensus_queue.cc:1243] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:33.263676 30427 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.257170 30463 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 12:19:33.300305 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:33.301757 30388 consensus_meta.cc:287] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.002s
I20260709 12:19:33.283404 30462 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:33.303050 30476 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.424s	user 0.002s	sys 0.000s
I20260709 12:19:33.284953 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.284119 30398 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:33.304751 30356 raft_consensus.cc:2764] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:19:33.305022 30398 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.305555 30398 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.306941 30441 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:33.307682 30471 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.416s	user 0.001s	sys 0.000s
I20260709 12:19:33.307597 30441 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.276243 30442 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.286355 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.275507 30175 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:33.311051 25631 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:33.312011 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:33.312428 30490 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:33.312623 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.312799 25631 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:33.313265 25779 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.287114 29987 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.276889 30433 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:33.265448 30430 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:19:33.287426 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.288471 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.316087 25632 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:33.316359 30438 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.290653 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.291330 30452 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:33.291857 30431 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.270445 30488 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:33.284637 30428 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.324479 30428 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.325220 25779 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:33.271154 30493 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:33.326224 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.315517 30417 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.289652 25634 raft_consensus.cc:1217] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.6->[11.7-11.7]   Dedup: 11.7->[]
I20260709 12:19:33.328826 25634 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:19:33.314282 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.312073 25784 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:19:33.320641 25636 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:19:33.297415 30473 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.318686 30452 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:33.322180 30465 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:33.328002 30463 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.331889 30356 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.319621 30237 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:33.333276 30494 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:33.335004 30322 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.338135 26335 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:33.339740 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.354048 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.375165 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.355917 25677 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:33.302248 30388 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.369683 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.322795 30425 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:19:33.370930 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:33.371582 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69961.5us
I20260709 12:19:33.371924 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.376605 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.378422 25632 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:33.370841 25780 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:33.394779 25786 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:33.395586 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.380362 25784 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:33.380554 30473 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:33.369222 30433 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.392597 30388 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.384927 25633 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:33.386646 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.388418 25779 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:33.391388 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.368413 30463 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:33.381803 30241 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.001s
I20260709 12:19:33.398664 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.408371 25780 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 12:19:33.378574 25631 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:33.395087 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:33.404240 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.435181 25632 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:33.431043 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.427785 30465 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.428251 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:33.445513 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50516.1us
I20260709 12:19:33.446710 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.447623 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:33.447597 25786 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:33.451898 26307 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:33.452749 30281 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:33.452829 30428 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.453434 30423 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.464154 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2026) took 2214 ms. Trace:
I20260709 12:19:33.464735 25783 rpcz_store.cc:276] 0709 12:19:31.249361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.423353 (+173992us) service_pool.cc:224] Handling call
0709 12:19:31.469557 (+ 46204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.183801 (+1714244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.463784 (+279983us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54002b4654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:33.464040 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3858278656}
I20260709 12:19:33.469383 30455 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:33.435535 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:33.428900 30452 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.470053 30455 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.475070 30420 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.475731 30281 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:33.475898 30420 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.476934 30281 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.436725 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.478127 25787 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:33.479034 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.440205 30463 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.373730 30356 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.508136 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.508910 25785 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:33.440959 30441 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.399325 30473 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.442718 30481 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.442909 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.443392 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.523975 25784 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:33.443647 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:33.444476 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.436256 25636 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:33.473158 25783 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:33.538897 25634 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:19:33.473977 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.539872 25637 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.437852 30390 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.407153 30433 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.445619 30241 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.446466 30432 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:33.496737 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.500484 25632 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:33.475101 30494 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:19:33.509949 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.510344 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2750) took 1096 ms. Trace:
I20260709 12:19:33.511256 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.554013 25711 rpcz_store.cc:276] 0709 12:19:32.401994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.411877 (+  9883us) service_pool.cc:224] Handling call
0709 12:19:32.412251 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:33.053950 (+641699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.480726 (+426776us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400203354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:33.498009 (+ 17283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440754560}
I20260709 12:19:33.512470 30435 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.564402 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.495862 30455 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.523411 30473 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:33.545248 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.548861 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:56416 (request call id 5) took 1001 ms. Trace:
W20260709 12:19:33.511540 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76180.8us
I20260709 12:19:33.571090 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:33.557482 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46353.5us
I20260709 12:19:33.512117 30493 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:19:33.546586 30228 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.572880 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.574391 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.570361 25630 rpcz_store.cc:276] 0709 12:19:32.537798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.545164 (+  7366us) service_pool.cc:224] Handling call
0709 12:19:33.523011 (+977847us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400364a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5995735 00007f29b598d9fa 00007f29ba9554c9 00007f29b3ed1c54 00007f29b3ed1be2 00007f29b3ed1b11 00007f29b3ed1a8c 00007f29b3ed0d42 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:33.538598 (+ 15587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424259840}
I20260709 12:19:33.575546 25779 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:33.538527 30463 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:33.548686 30356 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.588109 25637 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:33.549925 30492 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:33.584419 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:33.552434 30388 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.597470 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.603842 25636 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:33.605747 25784 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 7.
I20260709 12:19:33.606711 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.606863 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.607486 25784 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:33.610579 25521 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0ae979356d694a09be4b0cd976ea354a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:33.546000 30310 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:19:33.609391 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:33.571682 30501 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:33.605062 25677 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:33.595829 30301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.026s	sys 0.024s
I20260709 12:19:33.565152 30155 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:33.568921 30481 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:33.574219 30420 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.581764 30441 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:33.600178 30438 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:33.618415 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.591184 30455 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:33.622828 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.553733 30488 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.564512 29901 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:19:33.628999 25637 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:19:33.614310 30356 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:33.574658 30435 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.611997 30466 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:33.623302 30330 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:33.582971 30464 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.658136 26339 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
W20260709 12:19:33.621297 30301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.026s	sys 0.025s
I20260709 12:19:33.614651 30456 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:19:33.600822 30228 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.613512 30388 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.621691 30481 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.635028 30396 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:19:33.665222 26342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1111) took 1456 ms (client timeout 1882 ms). Trace:
I20260709 12:19:33.665973 25778 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:33.628623 30501 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:33.676550 26342 rpcz_store.cc:269] 0709 12:19:32.203693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.203872 (+   179us) service_pool.cc:224] Handling call
0709 12:19:33.659511 (+1455639us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54006702d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:33.659628 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135294720}
I20260709 12:19:33.618306 30405 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.651862 30356 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:33.678457 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.658109 30420 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:33.656111 30281 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.620982 30398 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.662469 30330 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.627897 30155 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.693549 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.610611 30492 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.705019 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:33.705650 25787 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:33.678049 30481 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:33.707015 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:33.710420 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 92073.7us
I20260709 12:19:33.694273 30356 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:33.662734 30441 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.704260 30330 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:33.702793 30396 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:33.715216 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:33.723767 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:33.724704 25784 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:33.708259 30435 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.729346 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:33.732014 26339 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.713861 30466 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:33.739290 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.740000 25784 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.741658 30456 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:19:33.743336 26341 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:33.705529 30281 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:33.748790 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 111507us
I20260709 12:19:33.754244 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.755607 26342 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:33.759903 25784 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:33.719743 30341 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:19:33.762275 25787 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:33.766661 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:33.768187 25784 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:33.737993 30466 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:33.734935 30390 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.776635 26339 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:33.739426 30513 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:33.742403 30433 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.779527 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:33.783947 26341 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:33.784871 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.785863 26339 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:33.787129 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.791016 25784 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:19:33.802989 30511 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:19:33.805034 30388 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.799857 30513 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:33.815070 30513 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.773125 30466 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:33.828847 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.830806 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:33.832806 26342 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:33.841617 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.842787 26335 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
W20260709 12:19:33.844413 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 80223.7us
I20260709 12:19:33.847888 25784 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:33.851292 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 130043us
I20260709 12:19:33.856750 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:33.858201 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:33.860198 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.849138 30100 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.865537 25780 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:33.866225 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:33.867457 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.865276 26339 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:33.868585 25784 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:33.851670 30488 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.873842 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.877863 26342 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:33.857820 30228 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.878652 26342 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:33.774020 30501 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.864181 30388 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:33.870944 30494 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:19:33.873034 30464 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.890939 30492 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.861912 30460 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:19:33.892896 25784 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:19:33.877640 30405 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:33.863916 29901 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:19:33.892149 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1114) took 1295 ms. Trace:
I20260709 12:19:33.895109 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:33.916406 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.893428 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:33.917193 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:33.889477 26342 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.888314 30462 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:19:33.920663 25635 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:33.881541 30241 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:33.838317 30396 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.916270 26338 rpcz_store.cc:276] 0709 12:19:32.596534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.596710 (+   176us) service_pool.cc:224] Handling call
0709 12:19:33.891949 (+1295239us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400582ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:33.892089 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791503232}
I20260709 12:19:33.915428 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:33.918125 26341 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:33.918669 26342 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:33.867913 30430 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:19:33.875141 30100 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:33.923003 25632 raft_consensus.cc:2368] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:33.886173 30398 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.936534 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:33.926030 26335 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:33.897670 30494 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:33.943734 26338 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:33.944630 26341 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:19:33.925637 30423 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:33.924016 30155 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.965484 26339 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:33.965678 26342 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:19:33.966535 26342 pending_rounds.cc:85] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 8
I20260709 12:19:33.966890 26342 pending_rounds.cc:107] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:19:33.967260 26342 raft_consensus.cc:2889] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:33.970659 26338 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:19:33.936281 30301 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:19:33.940804 30321 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:19:33.938829 30438 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:19:33.920182 30473 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:33.965142 30423 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.991240 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:33.991616 30100 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:33.991873 25711 raft_consensus.cc:2376] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14. Re-sending same reply.
I20260709 12:19:33.992897 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:33.996166 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:33.997166 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:33.998400 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:33.999161 25710 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:33.999886 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.000563 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:34.006057 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.007033 30385 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:34.007462 30385 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:34.007848 30385 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:34.008018 25706 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:34.008677 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.009068 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.010795 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.011327 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.011801 25707 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:34.012733 25708 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:19:34.016454 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.016562 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:34.017117 25710 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:34.023450 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 103720us
I20260709 12:19:33.992244 30423 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:34.030350 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.031050 25705 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:34.029695 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:19:34.023754 30518 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:19:34.037456 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.037643 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.025758 30420 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:34.045715 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.045938 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:34.045068 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.049571 30482 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:34.041975 30494 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.053316 30473 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.041797 30398 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:19:34.027073 30100 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:19:34.026393 30241 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:19:34.055573 30389 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.039773 30417 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:34.041596 30435 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:19:34.057194 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.057507 30420 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:34.057808 30483 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:34.058925 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.027173 30522 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:19:34.069468 25705 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:34.090207 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 948) took 1167 ms. Trace:
I20260709 12:19:34.090775 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.071053 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:34.073608 25706 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:34.005911 30466 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.037443 29901 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:34.075250 25710 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:34.077980 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.086930 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.078953 30417 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.091087 26336 rpcz_store.cc:276] 0709 12:19:32.922603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.922846 (+   243us) service_pool.cc:224] Handling call
0709 12:19:34.058755 (+1135909us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b5800154598. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:19:34.090139 (+ 31384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8018688}
I20260709 12:19:34.082012 30310 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:34.102000 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:34.102914 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:34.105172 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.104820 30420 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.106472 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.087034 30483 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:34.075467 30155 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:19:34.107091 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.096217 30526 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:19:34.096585 29901 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.112329 25706 raft_consensus.cc:2368] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:34.107643 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.097987 30417 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:34.116094 30483 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.110267 30522 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:34.109859 30310 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.099521 30463 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:34.118846 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.114176 30529 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:34.107069 30301 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:34.114449 30526 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:34.124037 30466 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.109299 30389 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:34.127247 30481 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.150840 25630 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:34.128207 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.133481 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.097589 30321 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:34.162032 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:34.143519 25708 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:34.186312 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.188977 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:34.141429 30310 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.193636 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.194981 30433 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:34.145820 29901 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.126838 30330 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.207237 30385 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.208880 30330 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:34.209599 30385 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.144049 30483 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:34.147747 30356 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.213231 30356 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.201694 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.200261 30463 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:34.216101 25752 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:34.214258 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 111387us
I20260709 12:19:34.133358 30405 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:34.143692 30482 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:34.218648 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:34.184687 30389 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:34.218545 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.145732 30526 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.236105 25631 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:19:34.232779 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.202888 30531 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:34.221737 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.222697 25711 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: []
I20260709 12:19:34.150420 30481 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.143626 30529 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:34.147398 30281 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.192675 30522 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:19:34.199255 30390 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:34.217577 30463 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.262763 30463 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:34.264446 29901 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.265653 30522 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:34.247908 30531 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:34.267022 26337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1108) took 2149 ms (client timeout 2212 ms). Trace:
I20260709 12:19:34.267212 30531 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:34.267534 26337 rpcz_store.cc:269] 0709 12:19:32.117570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.124157 (+  6587us) service_pool.cc:224] Handling call
0709 12:19:34.266738 (+2142581us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540061c454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:34.266928 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2555043200}
I20260709 12:19:34.269074 26344 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:34.269886 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.271287 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.272509 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.220505 25752 leader_election.cc:400] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:34.279258 25752 leader_election.cc:403] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:19:34.281636 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61348.6us
I20260709 12:19:34.249936 30535 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:34.254666 30530 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:34.271337 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 950) took 1343 ms. Trace:
I20260709 12:19:34.245008 26340 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:34.273831 30388 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:34.266289 30522 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.273626 25631 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:34.255923 30390 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.276528 30128 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:34.276749 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.273633 30529 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.243039 30482 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:34.277984 25707 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:34.257229 25638 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:34.287902 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.228825 30330 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:34.286533 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.292157 26344 rpcz_store.cc:276] 0709 12:19:32.928373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.928700 (+   327us) service_pool.cc:224] Handling call
0709 12:19:34.268861 (+1340161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.269493 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.271120 (+  1627us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540061c450. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:34.271267 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010416000}
I20260709 12:19:34.328699 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.329413 26344 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.295109 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.296012 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.298084 25635 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:34.302225 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.332548 25631 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:34.275729 30416 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.302675 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.304153 30530 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.336221 30529 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:34.337174 25631 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:34.337764 30530 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.338553 25631 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:34.316109 30417 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.298256 30501 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:19:34.342763 25631 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:34.342892 30522 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.294617 30389 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:34.344270 30389 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:34.309762 30455 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.664s	user 0.009s	sys 0.004s
I20260709 12:19:34.294456 30388 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:34.347283 30388 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.348546 30388 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:34.344518 30423 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.313607 30390 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:34.346413 30389 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.316604 25629 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:19:34.333649 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.334936 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.338966 30416 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.358129 30416 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:34.306746 29901 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.359704 30423 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.292675 30466 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:19:34.342367 30417 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.298211 30083 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:19:34.360782 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.361169 30433 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:34.361586 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.350685 30455 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:34.361902 30531 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:34.374419 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.363616 30390 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.381263 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.382160 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:34.382112 25785 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:34.382522 25638 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:34.382961 26342 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:34.390637 25706 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:34.363178 30482 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.398020 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:34.399458 26344 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:34.370958 30528 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.448812 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.460680 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.464223 25786 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:34.462984 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.529256 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.530088 25779 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
W20260709 12:19:34.443081 25781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2530) took 1738 ms (client timeout 2012 ms). Trace:
W20260709 12:19:34.531401 25781 rpcz_store.cc:269] 0709 12:19:32.704266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.704424 (+   158us) service_pool.cc:224] Handling call
0709 12:19:34.442832 (+1738408us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b54001a63d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:34.442994 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58778880}
I20260709 12:19:34.395819 30385 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:34.444665 25782 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:34.464555 25778 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:19:34.449626 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.439973 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2763) took 1395 ms. Trace:
I20260709 12:19:34.421963 30390 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.455226 30441 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.427248 29964 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:34.463178 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:34.545737 25784 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:34.365478 30463 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.428484 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 835) took 1034 ms. Trace:
I20260709 12:19:34.445015 26342 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:34.426313 30474 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 1.547s	user 0.000s	sys 0.007s
I20260709 12:19:34.464906 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:34.400233 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.550981 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.520807 25632 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:34.548985 26335 rpcz_store.cc:276] 0709 12:19:33.394166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.880693 (+486527us) service_pool.cc:224] Handling call
0709 12:19:33.922430 (+ 41737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.945279 (+ 22849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.428226 (+482947us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b5400471650. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:34.428409 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75827456}
I20260709 12:19:34.553292 25783 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:34.525341 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.554005 25780 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
W20260709 12:19:34.526254 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.534699 25709 rpcz_store.cc:276] 0709 12:19:33.044103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.070712 (+ 26609us) service_pool.cc:224] Handling call
0709 12:19:33.071119 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.246841 (+175722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:33.246883 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:33.246926 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:33.246956 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.439743 (+1192787us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400203854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:34.439791 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:34.439915 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403590400}
I20260709 12:19:34.553192 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.449658 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.558650 25706 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:34.559351 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.559990 25780 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:34.562503 25705 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:19:34.565147 25780 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:34.563766 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.565179 25786 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:34.570292 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2398) took 2812 ms. Trace:
I20260709 12:19:34.570818 25629 rpcz_store.cc:276] 0709 12:19:31.757568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.762935 (+  5367us) service_pool.cc:224] Handling call
0709 12:19:31.791451 (+ 28516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.354739 (+2563288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.569953 (+215214us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b54001d7ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:34.570185 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5806733184}
I20260709 12:19:34.576714 30463 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.560258 30441 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 12:19:34.579460 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 181475us
I20260709 12:19:34.582300 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.582324 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:34.587450 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 137892us
I20260709 12:19:34.592556 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:34.593751 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:34.588408 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.600100 26307 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.600850 25783 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:34.566601 30100 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:34.545521 30528 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.578378 30531 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.549985 30385 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.584199 30310 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:34.579276 30423 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:34.584156 30483 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:34.592315 30465 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:34.602322 30238 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:19:34.612879 30528 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.613835 30385 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.614526 30483 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.616734 30483 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.619654 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2028) took 1892 ms. Trace:
I20260709 12:19:34.620129 25782 rpcz_store.cc:276] 0709 12:19:32.726784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.752474 (+ 25690us) service_pool.cc:224] Handling call
0709 12:19:34.444235 (+1691761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.533266 (+ 89031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.619412 (+ 86146us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54001a63d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:34.619579 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3890026368}
I20260709 12:19:34.629562 25784 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:34.604118 30281 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 12:19:34.634434 30478 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.871s	user 0.000s	sys 0.011s
I20260709 12:19:34.635635 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2101) took 2357 ms. Trace:
I20260709 12:19:34.605576 30545 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:19:34.604280 30501 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:34.593060 30463 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:19:34.657002 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.657899 25636 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:34.636109 25633 rpcz_store.cc:276] 0709 12:19:32.278452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.279651 (+  1199us) service_pool.cc:224] Handling call
0709 12:19:32.323892 (+ 44241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.399503 (+1075611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.635349 (+1235846us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002ac1d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:34.635562 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2439244672}
I20260709 12:19:34.660288 25784 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:34.649710 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:34.652401 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.678098 25782 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 7: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:34.662829 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:34.685808 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:34.690685 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 79436.8us
I20260709 12:19:34.687166 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.709430 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.713968 25708 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:34.713727 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:34.681741 30426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.797s	user 0.009s	sys 0.023s
I20260709 12:19:34.718797 25636 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:34.721911 25782 raft_consensus.cc:2368] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:19:34.685189 30395 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:34.656576 30465 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:34.656689 30501 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.731942 30501 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:34.732404 30465 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.657480 30528 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.708078 30385 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:34.734760 30528 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.734905 30385 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.676134 30531 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.736948 30385 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.737231 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:34.737327 30531 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.739741 30531 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.736948 30528 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.737509 30385 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:34.744130 30385 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:34.732414 30543 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:19:34.636124 30416 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:19:34.746156 30432 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:34.747167 30432 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:34.747516 30432 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:34.746264 30385 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.636428 30100 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:19:34.685464 30441 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:34.636533 30423 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.756070 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.708087 29964 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:34.772053 29964 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:34.773132 29964 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.758543 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.778366 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.779776 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:34.780377 25782 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:34.721550 30426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.836s	user 0.009s	sys 0.023s
I20260709 12:19:34.768306 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.771473 30423 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:34.783298 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.708096 29962 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:19:34.794541 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.795537 30281 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:34.795719 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.796226 30281 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:34.797327 25779 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:34.754669 30395 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:34.800940 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2538) took 1297 ms (client timeout 1637 ms). Trace:
I20260709 12:19:34.801429 30395 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:34.804277 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:34.804554 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.802213 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.791404 29964 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:34.748384 30432 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.786689 30441 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:34.812516 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.797564 30281 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.813570 25779 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:19:34.817062 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.830905 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:34.831759 25779 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.818768 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:34.822939 25782 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
W20260709 12:19:34.801339 25785 rpcz_store.cc:269] 0709 12:19:33.503629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.572558 (+ 68929us) service_pool.cc:224] Handling call
0709 12:19:34.800749 (+1228191us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54002b5054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:34.800878 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812798720}
I20260709 12:19:34.834682 25629 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:34.836458 25782 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:34.845206 25751 leader_election.cc:400] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
W20260709 12:19:34.850231 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 261930us
I20260709 12:19:34.853012 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:34.856688 25781 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:34.816416 30440 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:19:34.862725 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 282268us
W20260709 12:19:34.866039 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.867552 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.868675 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.869239 25708 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:34.870026 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.880916 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.881673 26344 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:34.883033 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.884869 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.887413 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.829196 30546 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:34.890153 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.891055 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.891786 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:34.892838 26336 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:19:34.890300 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 200203us
I20260709 12:19:34.854074 30420 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:34.861153 30440 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:34.871099 30237 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:34.887488 30274 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.908183 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.912612 30190 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:34.926030 25712 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:34.924755 25781 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:34.901710 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:34.925118 25779 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:34.900609 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.929296 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.929481 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.905129 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.931455 25633 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:34.909363 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.930405 25636 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:34.940754 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 220677us
I20260709 12:19:34.909061 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:34.945675 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:34.946019 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:34.958164 25779 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:34.959259 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:34.959435 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50483.4us
I20260709 12:19:34.960368 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.889955 29901 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:19:34.961401 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:34.961691 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.961512 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:34.900043 30551 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:19:34.895569 25676 leader_election.cc:400] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:19:34.972491 25676 leader_election.cc:403] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:34.972929 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.973538 25781 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:34.899514 30281 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:34.975471 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:34.977407 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:34.978324 25677 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:34.933676 30423 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.010701 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:35.012213 25781 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:35.013656 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 758) took 1021 ms. Trace:
I20260709 12:19:35.014147 25711 rpcz_store.cc:276] 0709 12:19:33.992629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.994656 (+  2027us) service_pool.cc:224] Handling call
0709 12:19:34.007274 (+ 12618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.255547 (+248273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.013449 (+757902us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400224cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:35.013601 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494916864}
I20260709 12:19:35.019461 25631 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:34.963210 30416 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.963798 30398 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:35.032681 25778 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:19:35.032902 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49058.4us
I20260709 12:19:35.033514 25786 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:34.925216 30274 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.035792 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:34.981349 30452 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:34.980785 30190 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:35.075371 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:34.980365 30385 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:35.069620 25778 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:19:34.966603 30395 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:19:35.084581 25712 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:35.086885 25706 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:35.090641 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.073612 30531 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.030810 30440 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.036913 30423 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.046018 30416 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:34.934499 30441 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:34.981424 30501 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:35.099474 30552 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:19:35.110598 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.030090 30345 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:35.099946 30398 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:19:35.103494 30483 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:19:35.122156 30297 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:35.125617 25633 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:35.095749 30155 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:19:35.104964 30385 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:35.113565 30482 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:35.132632 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 255945us
I20260709 12:19:35.141498 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:35.152205 30530 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:35.152804 30530 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:35.099027 30564 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:19:35.122509 30345 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:35.102905 30531 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.126112 30567 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:35.168534 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:35.157176 30482 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:35.178179 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:35.139503 30297 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.186177 30572 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:35.187153 30567 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:35.187935 30567 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:35.186721 30572 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:35.190084 30297 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:35.197016 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 218805us
W20260709 12:19:35.198083 30554 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:19:35.204679 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:56416 (request call id 6) took 1057 ms. Trace:
I20260709 12:19:35.205163 25634 rpcz_store.cc:276] 0709 12:19:34.147156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.147321 (+   165us) service_pool.cc:224] Handling call
0709 12:19:35.204278 (+1056957us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b5400364a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5995735 00007f29b598d9fa 00007f29ba9554c9 00007f29b3ed1c54 00007f29b3ed1be2 00007f29b3ed1b11 00007f29b3ed1a8c 00007f29b3ed0d42 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:35.204500 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703935872}
W20260709 12:19:35.198020 30559 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:19:35.198050 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:35.209254 30553 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:19:35.209563 25677 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:35.210572 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 760) took 1217 ms. Trace:
I20260709 12:19:35.210896 30575 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:35.210959 25713 rpcz_store.cc:276] 0709 12:19:33.993221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.993702 (+   481us) service_pool.cc:224] Handling call
0709 12:19:33.993977 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:33.994302 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:33.994329 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.012727 (+ 18398us) log.cc:844] Serialized 26 byte log entry
0709 12:19:34.568705 (+555978us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b180029f020. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:19:34.847291 (+278586us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:34.847374 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.847523 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.210241 (+362718us) raft_consensus.cc:1685] finished
0709 12:19:35.210264 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.210506 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3848,"raft.queue_time_us":2594,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":6956,"spinlock_wait_cycles":3218688,"thread_start_us":278227,"threads_started":1,"wal-append.queue_time_us":892549}
I20260709 12:19:35.176484 30482 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:35.195196 30555 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:19:35.184126 30564 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:35.153905 30530 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:35.227877 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 152624us
I20260709 12:19:35.238154 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:35.242892 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:35.239573 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:35.257929 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:35.217600 30575 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:19:35.263166 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.272727 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:35.273667 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:35.274317 30462 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:35.291630 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:35.260393 30465 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:19:35.295403 25704 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:19:35.240336 30556 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 12:19:35.278177 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2533) took 1807 ms (client timeout 2016 ms). Trace:
I20260709 12:19:35.253336 30297 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:35.229883 30190 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:35.298135 25787 rpcz_store.cc:269] 0709 12:19:33.470380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.478738 (+  8358us) service_pool.cc:224] Handling call
0709 12:19:35.277951 (+1799213us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54000de3d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:35.278100 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455205248}
I20260709 12:19:35.304966 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:35.230079 30398 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:19:35.235394 30155 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:19:35.309002 30426 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 12:19:35.310322 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 85295.5us
I20260709 12:19:35.307186 30465 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:35.317515 25784 raft_consensus.cc:2410] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:35.297894 30492 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.320719 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:35.321870 25785 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:35.330070 25778 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:19:35.331202 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 71218.9us
I20260709 12:19:35.226318 30385 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.347287 25677 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:35.348348 30463 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:35.356320 30563 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:19:35.358698 30462 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:35.359315 30462 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:35.360502 30462 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.237463 30572 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.376394 25710 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:35.377454 25713 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 12:19:35.380298 25713 pending_rounds.cc:85] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 11
I20260709 12:19:35.380585 25713 pending_rounds.cc:107] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:19:35.380872 25713 raft_consensus.cc:2889] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:35.304201 30493 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:19:35.311239 30576 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:35.388844 30297 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:35.319988 30416 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.336802 30274 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.348623 29964 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.306259 30190 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.393510 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:35.408684 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 103776us
W20260709 12:19:35.413420 30573 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:35.413750 30565 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.414361 30565 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.415779 30432 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:35.416687 30565 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:35.418924 30570 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:19:35.419831 30426 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:35.420703 30426 consensus_queue.cc:1243] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:35.374650 30462 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:35.423491 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:35.412184 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:35.430696 25636 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:35.347653 30492 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.401868 30576 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.437601 25629 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:35.437467 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:35.442822 25635 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:35.382241 30417 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:19:35.425931 30574 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:35.405269 30581 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:35.461680 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 68285.3us
I20260709 12:19:35.429548 30456 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:35.462044 30581 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:35.461575 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 143713us
W20260709 12:19:35.464470 30578 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:35.468288 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:35.420995 30567 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:19:35.493831 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:35.421520 29964 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.499608 29964 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:35.438349 30274 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:35.496372 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:35.500797 30274 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.438704 30576 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:35.441787 30575 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:35.514411 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.516980 30493 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.517920 30569 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:19:35.520241 30345 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:35.527510 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 137598us
I20260709 12:19:35.486696 30398 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:35.492359 30574 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.551237 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.487798 30581 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:35.546332 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:35.505752 30416 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:35.557595 30421 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:35.580842 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 68391.7us
I20260709 12:19:35.504819 30594 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 12:19:35.585985 30438 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:35.596161 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:35.564077 30421 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.504374 30297 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.549535 30493 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.596684 30575 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.608914 29901 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:35.609433 29901 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:35.609876 29901 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:35.612771 30083 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:35.613161 30083 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:35.616564 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.596058 30438 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:35.568754 30416 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.593051 30385 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:35.563100 30574 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:35.601032 30590 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:19:35.635865 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53684.3us
I20260709 12:19:35.614006 30083 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.649078 30190 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:35.692904 30438 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.694108 30421 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:35.694566 30438 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:35.693063 30589 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:19:35.719843 30574 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:35.680795 30274 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.724632 30581 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.681321 30441 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:35.725981 30482 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.726997 30441 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.684075 30600 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:35.692031 30492 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:35.733261 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 237051us
I20260709 12:19:35.711951 30609 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
W20260709 12:19:35.714301 30607 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:19:35.715830 30493 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.696861 30083 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:35.676451 30398 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:35.734066 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2059) took 1019 ms. Trace:
I20260709 12:19:35.701602 30395 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:35.755015 25781 rpcz_store.cc:276] 0709 12:19:34.714811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.012801 (+297990us) service_pool.cc:224] Handling call
0709 12:19:35.079395 (+ 66594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.080960 (+  1565us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:35.283232 (+202272us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.396863 (+113631us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.397513 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.397695 (+   182us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:35.399283 (+  1588us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.487867 (+ 88584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.487966 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.728407 (+240441us) raft_consensus.cc:1685] finished
0709 12:19:35.728429 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.733964 (+  5535us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25425,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":372,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":117,"thread_start_us":333805,"threads_started":4,"wal-append.queue_time_us":138208}
I20260709 12:19:35.736936 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:35.683259 30565 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 12:19:35.800812 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 199086us
I20260709 12:19:35.738469 30574 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 12:19:35.799736 30281 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.779s	user 0.011s	sys 0.002s
I20260709 12:19:35.766568 30345 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.764031 30594 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.763863 30575 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:35.836148 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.736968 30600 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.842599 30599 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:35.863780 25603 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:35.873028 25603 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:35.877427 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2063) took 1077 ms. Trace:
I20260709 12:19:35.877310 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:35.878399 25784 rpcz_store.cc:276] 0709 12:19:34.800328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.818379 (+ 18051us) service_pool.cc:224] Handling call
0709 12:19:35.877199 (+1058820us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b5400320854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:35.877364 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110999936}
I20260709 12:19:35.880236 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:35.880796 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:35.856390 30575 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:35.889698 25710 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:35.892285 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:35.894554 30608 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:19:35.897109 30600 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.898818 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 965) took 1599 ms. Trace:
W20260709 12:19:35.899403 30452 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.804s	user 0.013s	sys 0.000s
I20260709 12:19:35.899296 26340 rpcz_store.cc:276] 0709 12:19:34.299192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.301131 (+  1939us) service_pool.cc:224] Handling call
0709 12:19:34.301474 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:34.363092 (+ 61618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.363125 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:34.363798 (+   673us) log.cc:844] Serialized 52 byte log entry
0709 12:19:34.593731 (+229933us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:34.605618 (+ 11887us) log.cc:844] Serialized 14 byte log entry
0709 12:19:34.609120 (+  3502us) log.cc:844] Serialized 14 byte log entry
0709 12:19:34.632855 (+ 23735us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b18001a65c0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:35.143978 (+511123us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b18001a65c0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:35.313591 (+169613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.898432 (+584841us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b54006cbbd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:35.898519 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.898570 (+    51us) raft_consensus.cc:1685] finished
0709 12:19:35.898587 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.898748 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3814,"prepare.queue_time_us":38756,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":22308,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":5666,"spinlock_wait_cycles":210249728,"thread_start_us":398994,"threads_started":2,"wal-append.queue_time_us":363084,"wal-append.run_cpu_time_us":5378,"wal-append.run_wall_time_us":1005410}
I20260709 12:19:35.902578 25786 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:19:35.903342 30615 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:19:35.861570 30594 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.862967 30432 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:35.905483 30432 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.906306 30575 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:35.908187 30432 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:35.888494 30297 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:35.909695 30594 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:35.910231 30522 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.910987 25713 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:35.883546 30345 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:35.917822 30522 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:35.911505 30441 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.869494 30581 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.871894 30482 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:35.883701 30529 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:35.928813 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:35.964998 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 310513us
I20260709 12:19:35.922523 30416 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:35.972265 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:35.973060 26337 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:35.962729 30522 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:35.985534 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.985672 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:35.963117 30432 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:35.875734 30398 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.882668 30599 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.988006 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2130) took 1345 ms. Trace:
I20260709 12:19:35.986105 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:35.989743 25630 rpcz_store.cc:276] 0709 12:19:34.643005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.644003 (+   998us) service_pool.cc:224] Handling call
0709 12:19:34.684990 (+ 40987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.776012 (+ 91022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.776057 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:34.776111 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:34.776155 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.987776 (+1211621us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b54003719d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:35.987828 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.987951 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208786176}
I20260709 12:19:35.992267 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:35.995296 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:35.996423 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:35.992945 25708 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
W20260709 12:19:35.997555 30613 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.381s	user 0.000s	sys 0.003s
I20260709 12:19:35.998677 30574 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:35.999145 30574 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.999171 30575 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:35.999785 30574 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:36.000923 30576 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.001258 30575 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.002163 25637 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:19:36.005761 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:36.008103 30610 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:19:35.991304 30432 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.013446 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:36.014235 26335 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:36.025036 30576 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:36.040771 30611 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:36.047231 30238 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:36.048040 30238 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.048336 30599 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:36.049247 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.052306 30530 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:36.052553 30238 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.059752 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:36.059295 25521 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:19:36.062817 30238 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:36.063287 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:36.064334 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.064493 30482 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:36.054021 29901 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.065138 30576 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:36.066350 30576 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.067428 29901 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.068763 30576 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.064378 30529 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:36.069890 30574 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.070597 30529 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.065122 30482 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:36.064690 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.078969 30482 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.080241 30463 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:36.080729 30463 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:36.080927 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.081759 25637 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:36.082309 30440 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:36.082455 30576 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:36.083048 30576 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.085594 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:36.085803 25713 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:36.086922 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.087277 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2773) took 1447 ms. Trace:
I20260709 12:19:36.087961 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:36.088351 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.087668 25706 rpcz_store.cc:276] 0709 12:19:34.639567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.980264 (+340697us) service_pool.cc:224] Handling call
0709 12:19:34.980491 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.096958 (+116467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.087035 (+990077us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54003c91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:36.087218 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244232576}
I20260709 12:19:36.089592 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:36.090848 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.091902 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.092721 25638 raft_consensus.cc:2410] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:19:36.069929 30423 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.097057 25637 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:36.098261 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.099078 25630 raft_consensus.cc:2410] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:19:36.101912 30576 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.102762 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.103067 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2657) took 1656 ms. Trace:
I20260709 12:19:36.104668 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.103467 25709 rpcz_store.cc:276] 0709 12:19:34.446216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.560601 (+114385us) service_pool.cc:224] Handling call
0709 12:19:34.560903 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.612190 (+ 51287us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:34.869811 (+257621us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.219939 (+350128us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.223516 (+  3577us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:35.223591 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:35.223656 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.417141 (+193485us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b1800287f20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1
0709 12:19:35.420682 (+  3541us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b1800287f20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:35.425963 (+  5281us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b1800287f20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:36.102830 (+676867us) spinlock_profiling.cc:243] Waited 51.1 ms on lock 0x7b5400204254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:36.102882 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.103007 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13062,"prepare.queue_time_us":491665,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":101,"raft.queue_time_us":5611,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":6019,"spinlock_wait_cycles":163107456,"thread_start_us":602250,"threads_started":2,"wal-append.queue_time_us":436954}
I20260709 12:19:36.105345 25630 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:36.106783 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.108774 25603 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.111569 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:36.112284 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.114002 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.116838 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.117954 25705 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:36.151886 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.171551 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1061) took 1212 ms. Trace:
I20260709 12:19:36.171952 25633 rpcz_store.cc:276] 0709 12:19:34.959621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.095625 (+136004us) service_pool.cc:224] Handling call
0709 12:19:35.116157 (+ 20532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.131321 (+ 15164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.171259 (+1039938us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b54001d6fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:36.171486 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15846272}
I20260709 12:19:36.073902 30529 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.072440 30574 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.110554 30535 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:36.166860 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 144594us
I20260709 12:19:36.014066 30416 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:36.174755 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:36.176370 30582 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.187391 30581 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:36.187839 30582 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.181017 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:36.189342 26335 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:36.170778 30238 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:36.184995 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.151690 30576 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:19:36.162020 30463 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.197000 30463 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.194810 30238 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.197782 30238 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:36.187882 30581 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:36.202245 30581 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:36.174795 30482 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:19:36.205192 25630 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:36.186234 30565 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:19:36.212080 26335 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:36.174290 30423 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:19:36.167007 30281 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.218395 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.190253 30582 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.248375 30416 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:36.245532 30237 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:36.259243 30582 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.265939 30416 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:36.263550 30493 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:19:36.247658 30281 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.303506 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:36.304193 25638 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:36.308149 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.312860 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1060) took 2038 ms. Trace:
I20260709 12:19:36.313306 25631 rpcz_store.cc:276] 0709 12:19:34.274266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.345135 (+ 70869us) service_pool.cc:224] Handling call
0709 12:19:35.018934 (+673799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.020022 (+  1088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.312654 (+1292632us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b5400362c50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:36.312793 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548877056}
W20260709 12:19:36.314693 30602 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.699s	user 0.005s	sys 0.010s
I20260709 12:19:36.319482 30385 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:36.319952 30238 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.321127 30531 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.321990 25638 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:36.302279 30501 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.324947 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:36.326042 25631 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:36.327405 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.328048 30526 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:19:36.334725 30493 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:36.335613 30624 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.467s	user 0.001s	sys 0.000s
I20260709 12:19:36.335351 30493 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.319983 30385 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:36.339100 30493 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.340422 30385 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.341838 30385 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.343025 30238 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.358284 30546 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:36.359138 30546 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:36.360499 30425 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:36.361050 30425 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:36.368131 25631 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:36.375139 30531 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.381773 30531 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.384639 30531 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:36.388710 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.390631 30581 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.394433 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:36.398756 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.401050 30281 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.404639 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.405433 25631 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:36.362110 30425 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.409508 30281 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.448604 30546 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.458425 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.459362 25630 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:36.460467 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.461254 25635 raft_consensus.cc:2393] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:36.462317 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.463404 26335 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:36.462910 25630 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:36.425945 30566 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:36.442226 30425 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.477787 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:36.480232 26337 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:36.482151 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:36.483577 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.484150 25635 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:36.444717 30582 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:36.486804 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:36.488443 25706 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:36.487628 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:36.444603 30281 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.483322 30462 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:36.495070 30462 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.496240 30281 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.497153 30462 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:36.498171 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2065) took 1697 ms. Trace:
I20260709 12:19:36.498811 25786 rpcz_store.cc:276] 0709 12:19:34.800947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.039241 (+238294us) service_pool.cc:224] Handling call
0709 12:19:35.039699 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.902998 (+863299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.497852 (+594854us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b540035a2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:36.498088 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1983563776}
I20260709 12:19:36.500201 30546 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.480064 30566 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.502254 30432 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:36.508275 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:36.509406 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:36.514648 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.489503 30483 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.533686 30483 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.534785 30581 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.536662 30483 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.537731 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.543081 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:36.544754 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:36.547967 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:36.524394 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.529932 26342 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:19:36.550901 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.546898 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:36.553861 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.555270 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.557367 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.559769 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.560354 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.560583 25711 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:36.561321 25709 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:36.563369 25603 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:36.568492 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.570429 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.571023 25707 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:36.571358 25706 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:36.573305 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:36.574957 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:36.562453 30645 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:19:36.579200 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.547528 30432 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:36.570959 30595 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:36.550709 30582 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:19:36.569730 30646 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:19:36.551437 30586 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:36.565716 30494 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:36.585731 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.545380 30546 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:19:36.589080 25704 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:36.582880 30581 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:36.594311 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.585033 30595 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.597604 26341 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:19:36.598454 26341 pending_rounds.cc:85] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 6
I20260709 12:19:36.598788 26341 pending_rounds.cc:107] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:19:36.599046 26341 raft_consensus.cc:2889] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:36.588497 30432 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:36.596994 30238 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:36.599958 30416 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.599723 30595 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.601102 30432 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.596864 30438 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.603690 30438 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.605859 30438 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.599706 30600 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.606981 30600 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.598459 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.609364 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:36.594630 30586 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.591058 30582 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:36.610009 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:36.614596 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.618093 30462 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:36.616441 25704 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:36.622474 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:36.606077 30440 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:36.613480 30633 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:36.628051 25706 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:36.608304 30581 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:36.610819 30238 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:36.626549 30633 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.625638 30440 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:36.627159 29962 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:36.642328 25637 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:19:36.649351 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:36.650439 25634 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:19:36.651717 25634 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:36.654974 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:36.658074 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:36.658846 30586 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.644428 26335 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:36.662351 25633 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:19:36.664170 25635 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:36.667196 25637 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:36.667464 30591 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:36.668097 30591 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:36.669210 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 1.20094e+06us
I20260709 12:19:36.668994 30591 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.670117 25750 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.644271 30633 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.645378 29962 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.676535 25632 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:36.682664 30603 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:36.683048 30604 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:36.677942 25629 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:36.688180 30596 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.688509 30452 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.688684 30596 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.689085 30452 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.680368 29962 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:36.691128 25786 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:19:36.690228 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.696144 25786 pending_rounds.cc:85] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 7
I20260709 12:19:36.696543 25781 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:36.691571 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.696483 25786 pending_rounds.cc:107] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:19:36.691994 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.697917 25786 raft_consensus.cc:2889] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:36.698415 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49910.7us
I20260709 12:19:36.698801 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:36.699018 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:36.699470 25786 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.699860 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:36.700270 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.692662 25779 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:36.691991 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.702466 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.702673 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.702443 25784 raft_consensus.cc:2410] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:19:36.703686 25779 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:36.700377 25783 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:36.704704 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.704689 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.693015 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.711560 25778 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:36.711723 25782 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:36.713832 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:36.714372 25782 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.716567 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:36.721441 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.716785 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.729872 25783 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:36.734339 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.735044 25784 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:36.735915 25779 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:36.691645 30330 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:36.737178 30330 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.691215 30596 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.740650 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.739348 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:36.741822 25786 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:36.679064 30456 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:19:36.741670 25781 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:36.691512 30522 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:36.713922 30530 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:36.691375 30452 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.712579 30438 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:36.744104 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2071) took 1425 ms. Trace:
I20260709 12:19:36.748277 25782 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:36.722785 30483 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:36.751612 25787 rpcz_store.cc:276] 0709 12:19:35.318777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.318987 (+   210us) service_pool.cc:224] Handling call
0709 12:19:35.369382 (+ 50395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.435332 (+ 65950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.435362 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:35.435405 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:35.435440 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.743797 (+1308357us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b54001252d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:36.743858 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.744019 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151131904}
I20260709 12:19:36.677227 30582 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.689800 30595 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.681864 30581 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:36.760655 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 90625.2us
I20260709 12:19:36.754518 30483 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:36.771139 30483 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:36.690418 30440 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.775272 30440 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:36.750703 30522 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.768321 29962 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.766778 30416 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.780727 25780 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:19:36.790441 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 191873us
I20260709 12:19:36.795979 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:36.796921 26341 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:36.759313 30330 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:36.806703 30483 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:36.808305 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 172610us
I20260709 12:19:36.812496 30522 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.791719 30494 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:36.815821 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.817281 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:36.818678 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.818936 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:36.819515 25704 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:36.814936 25778 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:36.816011 30416 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.821620 30416 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.818476 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:36.809907 30653 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:19:36.814677 30594 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.813931 30494 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.825668 30432 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:36.814456 30128 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:19:36.833374 30440 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:36.845055 26341 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:36.850927 30654 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:19:36.840766 30631 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:36.855291 30631 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:36.855744 30631 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:36.856712 30440 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.859031 30440 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.861624 30633 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.839018 30388 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:19:36.837850 30155 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:19:36.848953 30432 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.825619 30626 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:19:36.870889 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:36.882838 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:36.885912 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 58132.7us
I20260709 12:19:36.886224 30238 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:36.839704 30594 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.888832 30667 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:36.892033 29987 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:36.900535 30667 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:36.907421 30564 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:19:36.902359 30385 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:19:36.911175 30432 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:36.900681 29987 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:36.928726 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:36.919942 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:36.929579 25632 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:36.946122 25632 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:36.897437 30416 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:36.953490 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 238308us
I20260709 12:19:36.955650 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:36.912012 30633 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.914477 30566 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:36.953163 30432 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.972908 30594 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.978185 25704 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:36.955049 30462 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:36.958560 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:36.987366 30617 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:36.988169 30617 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:36.990928 30617 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:36.991318 30572 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:36.992091 30594 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:36.992884 30594 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.975477 30581 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:36.912537 29987 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:36.979713 30483 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:36.983136 30661 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.223s	user 0.004s	sys 0.000s
W20260709 12:19:37.016131 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 108096us
I20260709 12:19:36.966264 30600 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.000101 30594 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:36.997313 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.020500 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:36.994300 30663 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:37.022707 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.023851 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.025318 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.025437 25783 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:37.026032 25787 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:36.996269 30581 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:37.027832 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.029740 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:36.984511 30432 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.028425 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.030537 25787 raft_consensus.cc:2376] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11. Re-sending same reply.
I20260709 12:19:37.029117 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:37.029381 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.034768 25783 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:37.035655 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.025417 30483 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:36.998163 30531 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:19:37.040989 25601 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.027200 29987 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:37.026301 30668 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:37.031229 30581 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:37.044018 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.042853 30438 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:19:37.036090 30535 raft_consensus.cc:2764] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:19:37.026170 30594 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 12:19:37.056088 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.048708 30617 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:37.041375 30663 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:37.061414 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.062013 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.062767 25636 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:19:37.064000 25633 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:37.064024 25631 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:37.034024 30388 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:19:37.049697 30440 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:37.070123 30637 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.788s	user 0.006s	sys 0.000s
I20260709 12:19:37.049259 30668 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:37.047129 30671 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:37.086632 30574 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.087210 30574 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.085423 30631 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.089177 30388 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:37.090847 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.087019 30668 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.103670 30388 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:37.104099 30388 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:37.104475 30388 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:37.104701 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 1.55838e+06us
I20260709 12:19:37.081645 30617 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:37.107448 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2073) took 1780 ms. Trace:
I20260709 12:19:37.092073 30631 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:37.109776 30581 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.115907 30581 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:37.117352 30432 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:37.089188 30619 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:37.118621 30671 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:37.119189 30595 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.119722 30671 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.120962 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.121630 25633 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:37.122920 25783 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:19:37.123165 29962 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.123919 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:37.124152 30619 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:37.124456 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.124650 30594 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:37.125484 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.126057 25781 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:37.112250 30592 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:37.127326 25786 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:37.142372 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.142704 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.143006 25786 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:37.143872 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.144906 25786 raft_consensus.cc:2410] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:37.125114 25782 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:37.111137 25785 rpcz_store.cc:276] 0709 12:19:35.327306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.327610 (+   304us) service_pool.cc:224] Handling call
0709 12:19:35.328091 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:35.348580 (+ 20489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.348706 (+   126us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:35.396586 (+ 47880us) log.cc:844] Serialized 52 byte log entry
0709 12:19:35.446448 (+ 49862us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:35.446842 (+   394us) log.cc:844] Serialized 14 byte log entry
0709 12:19:36.314098 (+867256us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:36.646080 (+331982us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b180042a7e0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:36.657489 (+ 11409us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.107105 (+449616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.107201 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:37.107244 (+    43us) raft_consensus.cc:1685] finished
0709 12:19:37.107257 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.107391 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2123234,"prepare.queue_time_us":21694,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":13975,"raft.queue_time_us":1361120,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":1527,"spinlock_wait_cycles":78945536,"thread_start_us":60259,"threads_started":3,"wal-append.queue_time_us":205573}
I20260709 12:19:37.135085 30456 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:37.135692 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 359657us
I20260709 12:19:37.147781 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.136233 30390 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.150863 25778 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [10.5-12.6]
I20260709 12:19:37.149107 25786 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:37.149241 25787 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:37.125324 30432 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:37.150967 25785 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:37.136323 30330 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:37.153230 25786 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:37.154693 30463 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:37.155992 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.160495 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.125094 30619 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.161775 30619 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:37.161947 30598 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:37.160501 30432 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.163307 30432 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:37.163527 30666 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.313s	user 0.000s	sys 0.003s
I20260709 12:19:37.156924 25781 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:37.166666 30667 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.167551 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.161782 30190 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:37.168036 30667 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.169404 25636 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 12:19:37.170951 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:37.171489 30665 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:19:37.172035 30600 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.172585 30600 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.175307 30463 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:37.175688 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.148517 30597 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:19:37.176513 25780 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:37.176344 30463 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.123183 30617 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.178742 30238 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:37.179385 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.161792 25750 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:37.183434 25750 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.186182 25780 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:19:37.189684 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.189695 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.191596 25778 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:37.194790 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 153872us
I20260709 12:19:37.197104 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.206990 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:37.209329 26339 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:37.199625 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:37.221086 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.221972 29962 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:37.222297 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.222573 29962 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:37.223132 29962 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.223925 25601 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.180053 30600 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.225545 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.218331 30597 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:37.232980 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.235601 26335 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:37.235592 25782 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:19:37.237944 25782 pending_rounds.cc:85] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 8
I20260709 12:19:37.238209 25782 pending_rounds.cc:107] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:19:37.238250 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.238564 25782 raft_consensus.cc:2889] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:37.239410 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:37.240172 26336 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:37.125180 30594 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.237715 30592 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:19:37.245522 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.246567 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:37.188576 29964 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:37.250608 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.255863 25778 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:37.257437 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:37.219727 30389 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:37.225342 30190 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.153292 30456 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.244006 30482 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:37.258291 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:37.180409 30390 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.239718 30483 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:37.238978 30494 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.187201 30330 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.239494 30598 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.273201 30662 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:37.280162 30668 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:37.302511 30662 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.303133 30662 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.305512 30456 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:37.302910 30668 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:37.306420 30668 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.306754 30598 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.307788 30594 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:37.308486 30673 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:19:37.309109 30598 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:37.259835 30513 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:19:37.277335 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.310791 30513 consensus_queue.cc:1243] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:37.311388 30513 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:37.311455 30522 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:37.311825 30513 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:37.236089 30423 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:19:37.270668 30190 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.285904 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.286139 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.254678 30605 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:37.257977 30595 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:37.252099 29964 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:37.316138 30330 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:37.316357 29964 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:37.270967 30597 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:37.318806 30597 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.319236 30597 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.294502 30494 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.324950 30390 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:37.271868 30482 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:37.326074 30482 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.328778 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.329600 25780 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:37.329787 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.330104 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:37.301894 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.331620 25783 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:37.311837 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.332587 25778 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:37.334023 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.334662 25630 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:37.337026 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.337697 25629 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:37.350425 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:37.351852 26342 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:37.352784 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.353557 25780 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:37.366328 25704 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:37.366837 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.367691 25706 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:37.372170 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.374650 29987 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.375108 29987 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.375612 30572 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:37.376235 30572 consensus_queue.cc:1243] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:37.377079 29987 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.377764 29987 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:37.378129 29987 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:37.379436 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 46574.4us
I20260709 12:19:37.382949 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.384466 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.386128 30574 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.387239 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.389837 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.390163 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.381001 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.390789 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.391660 30567 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:37.392093 30567 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.392465 30567 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.392637 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.393189 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.394800 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:37.396003 29987 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.396639 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.394201 25782 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:37.397260 25635 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:37.398105 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.398154 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.398847 25782 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:37.404304 30667 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:37.405297 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.407959 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.410383 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.412007 30577 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:37.412485 30577 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.412885 30577 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.410555 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.416647 30155 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:37.418285 30574 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:37.418751 30574 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:37.419718 30574 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.411520 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:37.423468 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 82088.8us
I20260709 12:19:37.425246 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.425995 25704 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:37.426090 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:37.427286 25707 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:37.427987 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.428853 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.429149 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:37.429316 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.430007 25706 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.431322 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:37.432008 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.432121 25710 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.437844 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.442139 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.442734 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:37.443001 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:37.443691 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.448623 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.360812 29901 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.449880 29901 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.452463 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.452678 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:37.453991 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.455252 25782 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:37.456581 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.457530 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:37.457634 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.458170 25710 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:37.458319 26339 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:37.458521 25782 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:37.459808 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.460425 25782 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:37.460533 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:37.461287 26339 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:37.461290 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.462736 25782 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:37.452607 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.463826 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:37.464709 25781 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:37.465620 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:37.360392 30452 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:37.350934 30494 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.474417 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:37.467568 26339 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:37.426676 30530 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:37.467317 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:37.468856 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.475618 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.476291 25706 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:37.478354 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.495994 26336 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:37.514191 25781 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:37.441138 30390 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:37.513641 25637 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:37.524109 25705 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:37.524921 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.525287 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:37.525597 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.513499 25782 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:37.526750 30586 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:37.527272 30586 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.527611 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:37.312870 30513 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.528735 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:37.532328 25521 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:19:37.536404 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 250581us
I20260709 12:19:37.537876 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.538411 25630 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:19:37.538941 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.540382 30595 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:37.541039 30595 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:37.526240 30494 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:37.543105 30494 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.543525 30494 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:37.542160 30595 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.527629 30586 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:37.533999 30567 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.548251 25711 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:37.551774 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.552898 30438 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:37.553558 30438 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:37.554055 30438 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.472728 30526 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:37.468214 30582 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:37.436210 30572 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:19:37.470794 30420 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:37.471146 30452 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.580103 30452 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:37.437084 30594 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:19:37.582664 30658 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:19:37.583007 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.584100 30390 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:37.584600 30390 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:37.585793 30390 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 4, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.471357 30463 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 12:19:37.589260 30463 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:37.589252 30390 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.589776 30463 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:37.590754 30463 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.594576 30577 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.596875 30577 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:37.597086 30680 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.055s	user 0.003s	sys 0.000s
I20260709 12:19:37.550216 29962 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.599138 29962 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:37.601475 30567 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:37.607338 30388 consensus_meta.cc:287] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.001s
I20260709 12:19:37.607807 30388 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.609958 30388 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.614274 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.614827 25713 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:37.542289 30546 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:37.574723 30462 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.615494 30600 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:37.616000 30600 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.615958 30462 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.616777 30600 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:37.617466 25637 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:37.565832 30685 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:37.565327 30693 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:19:37.622202 30190 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:37.624794 25712 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:37.626556 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:37.690788 25635 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:37.627292 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:37.628259 30674 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 12:19:37.631256 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 68276.4us
I20260709 12:19:37.631546 25786 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:19:37.698632 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 67318.8us
I20260709 12:19:37.573503 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:37.472590 30238 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:37.477707 30574 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:37.691727 25632 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:37.727558 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 154200us
I20260709 12:19:37.582182 30586 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:37.709189 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.740981 25632 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:37.742348 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.628043 30600 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.715544 25705 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:37.748208 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.720978 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.723435 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:37.694031 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:37.753865 25637 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.727808 30238 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:37.760025 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.765262 25637 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
W20260709 12:19:37.771015 25536 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 195) took 23339 ms (client timeout 15000 ms). Trace:
I20260709 12:19:37.776942 25635 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
W20260709 12:19:37.727986 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 91757.2us
I20260709 12:19:37.778679 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:37.784200 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 316929us
I20260709 12:19:37.784744 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.791126 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.729049 30526 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:37.740800 30128 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:37.814837 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.730743 30685 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:37.684358 30535 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:19:37.723986 29964 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.830001 25780 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:37.815145 30128 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:37.826349 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 47789.7us
I20260709 12:19:37.783392 30238 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.729600 30662 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:37.817451 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.782959 30531 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:37.837779 25635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:19:37.819972 25631 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:37.818133 25707 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:37.784305 30581 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:37.782546 30582 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:37.783777 30673 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.865307 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.866340 25631 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:37.840435 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:37.816514 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44327.3us
W20260709 12:19:37.774308 25536 rpcz_store.cc:269] 0709 12:19:14.456597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.462323 (+  5726us) service_pool.cc:224] Handling call
0709 12:19:37.746324 (+23284001us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:32.254873 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:32.387308 (+132435us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:32.387383 (+    75us) write_op.cc:435] Acquired schema lock
0709 12:19:32.387417 (+    34us) tablet.cc:662] Decoding operations
0709 12:19:32.389041 (+  1624us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:32.389154 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:32.389192 (+    38us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:19:32.587699 (+198507us) tablet.cc:701] Row locks acquired
0709 12:19:32.587759 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:19:32.587990 (+   231us) write_op.cc:270] Start()
0709 12:19:32.588177 (+   187us) write_op.cc:276] Timestamp: P: 1783599572587918 usec, L: 0
0709 12:19:32.588214 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:19:32.588979 (+   765us) log.cc:844] Serialized 7553 byte log entry
0709 12:19:34.424297 (+1835318us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:35.034634 (+610337us) op_driver.cc:464] REPLICATION: finished
0709 12:19:35.591847 (+557213us) write_op.cc:301] APPLY: starting
0709 12:19:35.597998 (+  6151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:36.015875 (+417877us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:36.017619 (+  1744us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:36.215548 (+197929us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:36.215992 (+   444us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 12:19:36.216057 (+    65us) write_op.cc:312] APPLY: finished
0709 12:19:36.216572 (+   515us) log.cc:844] Serialized 172 byte log entry
0709 12:19:36.292860 (+ 76288us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:36.299232 (+  6372us) write_op.cc:454] Released schema lock
0709 12:19:36.299327 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:36.299874 (+   547us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:36.299907 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":220,"mutex_wait_us":3670421,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":491296,"apply.run_cpu_time_us":53999,"apply.run_wall_time_us":724542,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":623812,"prepare.run_cpu_time_us":14141,"prepare.run_wall_time_us":456499,"raft.queue_time_us":340577,"raft.run_cpu_time_us":18628,"raft.run_wall_time_us":386324,"replication_time_us":2378228,"spinlock_wait_cycles":42568960,"thread_start_us":1082493,"threads_started":5,"wal-append.queue_time_us":387018,"wal-append.run_cpu_time_us":47576,"wal-append.run_wall_time_us":4003559}]]}
I20260709 12:19:37.860592 26307 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.870245 25603 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.874248 25787 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:19:37.874703 25637 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 12:19:37.863138 30597 consensus_meta.cc:287] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.000s
I20260709 12:19:37.853371 30128 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:37.814894 30425 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:19:37.861605 30581 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:37.836840 30462 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.882381 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.886439 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.892735 25629 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:37.887135 25780 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:37.895713 25636 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:37.892047 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:37.869099 30631 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:37.908773 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:37.923645 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.925665 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.882886 30535 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:37.934454 25631 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:37.935882 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:37.905604 30581 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:37.881496 30530 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:19:37.911453 30423 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:37.872920 30673 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.939040 30596 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:37.974326 30694 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:19:37.961719 30678 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.469s	user 0.001s	sys 0.000s
I20260709 12:19:37.932554 30483 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:37.886451 30597 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:37.975379 30681 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.414s	user 0.001s	sys 0.000s
I20260709 12:19:37.932770 30440 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:37.984741 30690 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 12:19:37.725203 30456 consensus_queue.cc:1175] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7bf8f20701245bcbaa09cc663025f97" responder_term: 12 status { last_received { term: 5 index: 5 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:37.904744 29962 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 12:19:37.979758 30691 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:19:37.960388 30416 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:37.950640 30631 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:37.962355 30566 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:37.988618 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:37.964599 30190 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:38.018338 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:37.963158 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:37.975679 29987 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.579s	user 0.003s	sys 0.007s
I20260709 12:19:38.021464 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:38.013432 30432 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.850s	user 0.009s	sys 0.003s
I20260709 12:19:38.003747 25632 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
W20260709 12:19:37.976332 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 117204us
I20260709 12:19:38.026747 25632 pending_rounds.cc:85] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 8
I20260709 12:19:38.027125 25632 pending_rounds.cc:107] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:19:38.027354 25632 raft_consensus.cc:2889] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:38.004904 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.921875 30462 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 12:19:38.012619 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 152173us
I20260709 12:19:38.000944 30600 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:37.937935 30526 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:37.912288 30494 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.025473 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:38.047987 25779 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:38.009145 30586 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:38.023947 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 60896.4us
I20260709 12:19:38.016768 30566 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:38.044001 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:38.035357 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 183470us
I20260709 12:19:38.050822 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:37.966136 30685 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.058166 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:38.061592 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:38.005473 30440 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.059515 30423 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:38.059100 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:38.029235 30522 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:38.051610 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 309314us
I20260709 12:19:38.060595 30596 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:38.078712 30600 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:38.078701 30703 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:38.079509 30390 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.077873 26307 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:38.080452 30696 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:19:38.075654 25779 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:38.094200 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:38.094420 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:38.028185 29987 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.095470 25633 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:38.096952 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:38.098901 25636 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:38.061127 30702 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:38.045130 30432 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.102950 25779 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:38.074694 30662 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:38.111078 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:38.021915 30704 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:19:38.051275 30697 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:19:38.057577 30438 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.061412 30190 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:19:38.117888 25633 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:38.092656 30425 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:38.123030 29901 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.089433 30703 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:38.120579 25636 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:38.126521 25677 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:38.103113 30535 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:38.134193 25708 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:38.109982 30522 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:38.135823 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:38.137038 25779 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:38.116094 30702 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.159349 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:38.161892 25633 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:38.120610 30420 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:19:38.169687 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 83888.4us
I20260709 12:19:38.176321 26341 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:38.176409 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:38.177165 26336 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:38.183267 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:38.134922 30703 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:38.119385 30596 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:19:38.204212 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:38.080504 30423 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.155997 30522 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.205379 26336 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:38.206629 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:38.117051 30710 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
W20260709 12:19:38.211035 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 132287us
I20260709 12:19:38.211625 26341 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:38.212532 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:38.169332 30702 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.106930 30631 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.180267 30709 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:38.219792 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:38.227003 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:38.232152 30535 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.117749 30566 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.233027 30190 consensus_queue.cc:1243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:38.214893 30522 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:38.218987 30586 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.217691 30597 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:38.251021 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:38.228269 30717 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:38.232515 30712 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:38.239243 30530 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:19:38.267248 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 131491us
I20260709 12:19:38.241015 30673 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.238611 30425 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:38.269217 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 96530.6us
W20260709 12:19:38.277873 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58183.3us
I20260709 12:19:38.278905 26307 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:38.161437 30708 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:19:38.284194 30440 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.261756 30720 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:38.285384 30720 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:38.271587 30699 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:19:38.261991 30586 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.289024 30586 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.259729 30704 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:38.299757 26339 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:38.303674 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:38.303828 30726 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:38.310878 30494 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:38.314203 30452 consensus_meta.cc:287] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.734s	user 0.010s	sys 0.001s
I20260709 12:19:38.326093 29987 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.328040 30128 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.316738 30704 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:38.336406 30238 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:38.352324 30668 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.038s	user 0.011s	sys 0.002s
I20260709 12:19:38.305467 30709 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:38.307947 30718 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:38.357375 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 86602.1us
I20260709 12:19:38.363705 26340 raft_consensus.cc:2368] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:19:38.361969 30714 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:38.369853 30704 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.357642 30438 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:38.375798 30720 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:38.377296 30619 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.215s	user 0.005s	sys 0.008s
W20260709 12:19:38.381068 30692 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.805s	user 0.011s	sys 0.000s
I20260709 12:19:38.358197 30718 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.381739 30718 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:38.387022 25709 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:38.354254 30432 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:38.334144 30702 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.357950 30709 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:38.429352 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:38.463660 30714 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:38.464190 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2103) took 1004 ms. Trace:
I20260709 12:19:38.464367 30714 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:38.464663 25787 rpcz_store.cc:276] 0709 12:19:37.459600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.600871 (+141271us) service_pool.cc:224] Handling call
0709 12:19:37.601192 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:37.875103 (+273911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.463907 (+588804us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b540035a2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:38.464128 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627337472}
I20260709 12:19:38.465576 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:38.465750 30714 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:38.456286 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.352926 30452 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:38.457929 26307 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.619s (SENT) [suppressed 13 similar messages]
I20260709 12:19:38.388037 30462 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:38.362010 29987 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:19:38.361624 30668 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.368381 30425 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.466173 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:38.662848 25787 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:38.638947 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:38.470777 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:38.663915 26337 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:38.664134 25629 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:38.466234 25636 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:38.667148 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:38.668154 30398 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:38.668716 25778 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:38.668118 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:38.670732 30452 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.668602 30398 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.671337 30398 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:38.473737 29964 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.459475 30432 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.676486 30432 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.619148 25704 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:38.680001 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:38.519855 30566 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:38.598099 30595 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.010s	user 0.013s	sys 0.000s
W20260709 12:19:38.689708 30595 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.101s	user 0.014s	sys 0.000s
W20260709 12:19:38.618690 30581 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.002s
I20260709 12:19:38.690927 30581 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.692495 25638 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:38.694892 30581 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.488659 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:38.741195 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:38.453907 30438 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.245s
I20260709 12:19:38.580313 25632 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:38.745297 30722 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:38.745770 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:38.747768 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:38.495012 25782 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:38.481020 25707 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:19:38.749960 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:38.492060 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 854) took 1168 ms. Trace:
I20260709 12:19:38.751119 25780 rpcz_store.cc:276] 0709 12:19:37.323225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.407186 (+ 83961us) service_pool.cc:224] Handling call
0709 12:19:38.491835 (+1084649us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54001257d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:38.491988 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938460544}
I20260709 12:19:38.752728 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:38.753964 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:38.515599 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2669) took 1150 ms. Trace:
I20260709 12:19:38.754593 25709 rpcz_store.cc:276] 0709 12:19:37.365577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.387000 (+ 21423us) service_pool.cc:224] Handling call
0709 12:19:38.515348 (+1128348us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54003ab454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:38.515486 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2297910016}
I20260709 12:19:38.755947 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:38.541477 30712 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.757333 30712 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.506654 30720 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.702093 30462 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:38.765669 30720 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.766839 30462 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.767751 30720 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:38.631222 30463 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:38.605551 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.490950 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 802) took 1059 ms. Trace:
W20260709 12:19:38.538564 30688 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.011s	user 0.012s	sys 0.003s
W20260709 12:19:38.462312 30711 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:19:38.517905 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2675) took 1128 ms. Trace:
I20260709 12:19:38.639981 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:38.414414 30717 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:38.779289 25712 rpcz_store.cc:276] 0709 12:19:37.389622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.390995 (+  1373us) service_pool.cc:224] Handling call
0709 12:19:37.391380 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.692143 (+300763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.517626 (+825483us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54003abe54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:38.517837 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536309632}
W20260709 12:19:38.622102 30729 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 12:19:38.594324 30574 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.865s	user 0.014s	sys 0.000s
W20260709 12:19:38.524187 30687 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.996s	user 0.010s	sys 0.011s
I20260709 12:19:38.608716 30703 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:38.736286 30734 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:19:38.488482 30727 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.532124 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 996) took 1068 ms. Trace:
W20260709 12:19:38.792330 30390 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.002s
I20260709 12:19:38.455463 30725 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.301s
W20260709 12:19:38.517228 30695 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.628s	user 0.002s	sys 0.000s
I20260709 12:19:38.761092 30712 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:38.819756 25710 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:38.764454 25780 raft_consensus.cc:2410] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:38.765951 25630 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:38.821026 30717 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:38.810442 30685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.013s
I20260709 12:19:38.767822 30462 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:38.719871 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:38.426625 30709 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.793833 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:38.777896 25708 rpcz_store.cc:276] 0709 12:19:37.431160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.453705 (+ 22545us) service_pool.cc:224] Handling call
0709 12:19:37.454015 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.157113 (+703098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.490672 (+333559us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b54002242d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:38.490878 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1552775552}
I20260709 12:19:38.608352 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:38.804190 26340 rpcz_store.cc:276] 0709 12:19:37.463354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.463502 (+   148us) service_pool.cc:224] Handling call
0709 12:19:38.531870 (+1068368us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54004970d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:38.532043 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2066621696}
I20260709 12:19:38.820055 30702 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:38.820235 30574 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:38.763805 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:38.806214 30482 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.480s	user 0.014s	sys 0.000s
I20260709 12:19:38.804477 29987 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.796856 30531 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:19:38.806010 30530 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:38.827090 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1145) took 1552 ms. Trace:
I20260709 12:19:38.806025 30128 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:38.827104 25712 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:38.829896 25787 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:38.859338 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:38.857998 26335 rpcz_store.cc:276] 0709 12:19:37.274810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.275257 (+   447us) service_pool.cc:224] Handling call
0709 12:19:37.275734 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:37.276153 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.276181 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:37.276679 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:19:37.435048 (+158369us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:37.435104 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.435204 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.523343 (+1088139us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:38.823394 (+300051us) raft_consensus.cc:1685] finished
0709 12:19:38.823417 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.826999 (+  3582us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":742,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":9925,"spinlock_wait_cycles":88231168,"thread_start_us":155880,"threads_started":1,"wal-append.queue_time_us":248647}
I20260709 12:19:38.826567 25629 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:38.811141 30390 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.671337 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:38.851123 25705 pending_rounds.cc:85] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 6
W20260709 12:19:38.823166 30685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.000s	sys 0.015s
I20260709 12:19:38.859572 25787 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:38.831480 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:38.863485 25705 pending_rounds.cc:107] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:19:38.867835 25705 raft_consensus.cc:2889] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:38.827265 30416 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.809s	user 0.015s	sys 0.001s
W20260709 12:19:38.863574 30631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.007s	sys 0.016s
I20260709 12:19:38.865471 25787 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.831612 30574 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:38.883606 25635 raft_consensus.cc:606] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:38.887688 25787 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:38.891855 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2182) took 1335 ms. Trace:
I20260709 12:19:38.896377 25630 rpcz_store.cc:276] 0709 12:19:37.556173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.558866 (+  2693us) service_pool.cc:224] Handling call
0709 12:19:37.559168 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.822018 (+1262850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.891569 (+ 69551us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003011d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:38.891791 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2774981504}
I20260709 12:19:38.881079 25636 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:38.808678 30727 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:38.905395 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 447505us
I20260709 12:19:38.883659 30416 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:38.886194 30631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.007s	sys 0.016s
I20260709 12:19:38.911744 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:38.913910 25712 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:38.856899 30482 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.888209 30432 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:38.905109 30423 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.013s	sys 0.009s
I20260709 12:19:38.862927 29987 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:38.865696 30530 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:38.976289 25636 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:38.978391 25630 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:38.978535 26336 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:38.982137 25779 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:38.888746 30128 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:19:38.990238 30530 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:38.990455 30513 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:38.991707 30567 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:38.991847 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1102) took 1172 ms. Trace:
I20260709 12:19:38.992164 30567 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:38.992336 25638 rpcz_store.cc:276] 0709 12:19:37.819016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.819199 (+   183us) service_pool.cc:224] Handling call
0709 12:19:37.825312 (+  6113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.692988 (+867676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.991612 (+298624us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54001c5cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:38.991790 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993796864}
I20260709 12:19:38.993105 30567 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:38.994632 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 87008us
I20260709 12:19:38.996032 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 846) took 1853 ms. Trace:
I20260709 12:19:38.996415 25785 rpcz_store.cc:276] 0709 12:19:37.142211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.174576 (+ 32365us) service_pool.cc:224] Handling call
0709 12:19:37.174933 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:37.178699 (+  3766us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:37.355781 (+177082us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.986088 (+630307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:38.002260 (+ 16172us) log.cc:844] Serialized 26 byte log entry
0709 12:19:38.002441 (+   181us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:38.002544 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.002632 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.516552 (+513920us) spinlock_profiling.cc:243] Waited 6.67 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:38.995760 (+479208us) raft_consensus.cc:1685] finished
0709 12:19:38.995865 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.995972 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26927,"prepare.queue_time_us":90042,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":47825,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":8597,"spinlock_wait_cycles":15352192,"thread_start_us":600984,"threads_started":3,"wal-append.queue_time_us":380264}
I20260709 12:19:38.999598 25785 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:39.000370 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2587) took 1072 ms. Trace:
I20260709 12:19:39.000942 25785 rpcz_store.cc:276] 0709 12:19:37.927953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.998377 (+1070424us) service_pool.cc:224] Handling call
0709 12:19:38.999278 (+   901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:39.000026 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.000293 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:39.004302 30727 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:39.005483 30668 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:39.007349 30703 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.010285 30482 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:39.010910 25631 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:39.011330 30128 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:39.011720 30128 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:38.936297 30388 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:19:39.012632 30388 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:39.013082 30388 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:39.012563 30128 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.013962 30128 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:39.016438 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:39.028439 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:39.028826 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:39.031477 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:39.047580 25787 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:39.048635 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2585) took 1212 ms. Trace:
I20260709 12:19:39.075986 25787 rpcz_store.cc:276] 0709 12:19:37.835849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.828929 (+993080us) service_pool.cc:224] Handling call
0709 12:19:38.829725 (+   796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:39.048243 (+218518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.048546 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:39.063671 26336 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:39.066615 25708 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:19:38.974833 30691 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:19:39.013846 30388 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.084908 25636 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:39.096195 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:39.010303 30713 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:39.107327 25711 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:39.080199 30574 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.066788 30546 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.066759 30577 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:39.146220 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60940 (request call id 0) took 1380 ms. Trace:
I20260709 12:19:39.186400 25635 rpcz_store.cc:276] 0709 12:19:37.783593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.784436 (+   843us) service_pool.cc:224] Handling call
0709 12:19:39.146029 (+1361593us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540021fa54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5989853 00007f29ba95760a 00007f29b3edd7d4 00007f29b3edd762 00007f29b3edd691 00007f29b3edd60c 00007f29b3edc8c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:39.146112 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364075776}
I20260709 12:19:39.076185 30432 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:19:39.102823 30713 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:39.142695 30388 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:39.113266 30522 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.853s	user 0.013s	sys 0.000s
I20260709 12:19:39.225219 25635 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:39.189356 30704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.041s	sys 0.051s
I20260709 12:19:39.148200 30238 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:39.234886 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:39.228490 30704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.056s	sys 0.058s
I20260709 12:19:39.212427 30546 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.239856 30546 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.241094 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:39.247661 30708 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.248327 30708 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:39.250649 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 952955us
I20260709 12:19:39.252424 30456 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:39.252830 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:39.253340 30708 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.253716 30658 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:39.253901 30708 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:39.254302 30658 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.254390 30708 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.252897 30456 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:39.256211 30456 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.256948 30452 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:39.257786 30456 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:39.258375 30398 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 12:19:39.220424 30483 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:39.253033 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 694524us
I20260709 12:19:39.254446 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:39.258870 30398 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.232632 30522 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.096277 30668 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.133s
I20260709 12:19:39.211045 30577 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.282195 30420 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.288184 30716 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.287927 30705 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.333226 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:39.293694 30708 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.295919 30390 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:39.311873 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.315155 25533 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:39.317786 30596 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:39.290269 30190 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.313553 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2579) took 1479 ms. Trace:
I20260709 12:19:39.292723 30732 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.339174 25781 rpcz_store.cc:276] 0709 12:19:37.833913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.834079 (+   166us) service_pool.cc:224] Handling call
0709 12:19:37.859517 (+ 25438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:38.005650 (+146133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.005685 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:38.006178 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:19:38.114326 (+108148us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:38.147693 (+ 33367us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.174903 (+1027210us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b18003dce40. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:39.293392 (+118489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.293765 (+   373us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.293815 (+    50us) raft_consensus.cc:1685] finished
0709 12:19:39.310598 (+ 16783us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.313436 (+  2838us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59855,"prepare.queue_time_us":40683,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":1104192,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":28771,"spinlock_wait_cycles":29713792,"thread_start_us":344092,"threads_started":3,"wal-append.queue_time_us":167096,"wal-append.run_cpu_time_us":2456,"wal-append.run_wall_time_us":1092747}
I20260709 12:19:39.282277 30483 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.330430 29964 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:39.356913 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:39.344321 30705 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.349025 25533 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 9 to 12, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 12 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:39.360030 30741 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:19:39.360790 30753 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:39.370528 30398 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.337771 30724 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.372611 30522 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.329782 30420 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.350641 30760 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:39.342564 30716 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.382180 25634 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:39.347730 30390 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.387149 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:39.383450 25533 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 7 to 11, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 11 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:19:39.357961 30463 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.001s
I20260709 12:19:39.309072 30713 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.374716 30761 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.342222 30596 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:39.346324 30732 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.383613 29964 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.407984 25533 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 6 to 10, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 10 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.436919 30763 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.348083 30190 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.460124 30190 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.428136 25781 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:39.394240 30758 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.397572 30724 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.401999 30760 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.445763 30577 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.446854 30763 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.429378 30432 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:19:39.448493 30483 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:39.449762 29964 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.422438 30762 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:39.451321 30718 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.069s	user 0.014s	sys 0.000s
I20260709 12:19:39.452019 30513 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:39.452947 25783 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:39.408697 30522 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.493093 30238 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.494076 26309 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:39.521591 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 164731us
I20260709 12:19:39.542358 30425 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:39.488701 30713 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.495591 30720 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.555641 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2105) took 2094 ms. Trace:
I20260709 12:19:39.455039 30708 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:39.502797 30190 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:39.553002 30128 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.539s	user 0.003s	sys 0.012s
I20260709 12:19:39.576344 25786 rpcz_store.cc:276] 0709 12:19:37.460963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.718044 (+257081us) service_pool.cc:224] Handling call
0709 12:19:37.718340 (+   296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:37.718732 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:37.860834 (+142102us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.361193 (+500359us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.298665 (+937472us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:39.337422 (+ 38757us) log.cc:844] Serialized 52 byte log entry
0709 12:19:39.337575 (+   153us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:39.337611 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.337678 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.555394 (+217716us) raft_consensus.cc:1685] finished
0709 12:19:39.555415 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.555577 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1020096,"prepare.queue_time_us":142995,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":1414,"raft.queue_time_us":1994891,"raft.run_cpu_time_us":501,"raft.run_wall_time_us":6008,"thread_start_us":477775,"threads_started":3,"wal-append.queue_time_us":348140}
I20260709 12:19:39.565585 30768 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.565888 30513 consensus_queue.cc:1243] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:39.565659 30597 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:39.594005 30709 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.001s
I20260709 12:19:39.431378 30761 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:39.568457 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 74467.1us
I20260709 12:19:39.555794 30718 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.573388 30698 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:39.571856 30425 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:39.464536 30758 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.598798 30597 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.612551 30705 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.592188 30190 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.596880 30390 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.564976 30522 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.580461 30708 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.614548 30708 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.583647 30720 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.615924 30522 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.556669 30762 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:39.610069 30757 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:19:39.588795 30420 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.611303 30718 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:39.613473 30758 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.597606 30452 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:19:39.572049 30238 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.602349 30709 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.625974 30760 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.552237 30596 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.627255 30596 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:39.621282 30732 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.599622 30768 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.630335 30761 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.623790 30763 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.624503 30425 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:39.625399 30566 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.943s	user 0.013s	sys 0.000s
I20260709 12:19:39.633088 30416 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.633227 30425 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:39.625399 30762 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:39.633724 30462 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.806s	user 0.012s	sys 0.001s
I20260709 12:19:39.633607 30416 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:39.631521 30767 consensus_queue.cc:1175] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7bf8f20701245bcbaa09cc663025f97" responder_term: 11 status { last_received { term: 10 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:39.616771 30720 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.638881 30709 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:39.636749 30708 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:39.536314 25520 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 60051e5723b2429582407076a003096c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 10d0c7349085463595aedaf7c071e742 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:19:39.672854 25520 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 136550us
I20260709 12:19:39.489289 25533 catalog_manager.cc:5697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 8, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 8 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.681978 25533 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:39.714419 25533 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:39.538653 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2187) took 1457 ms. Trace:
I20260709 12:19:39.739192 25631 rpcz_store.cc:276] 0709 12:19:38.081779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.084234 (+  2455us) service_pool.cc:224] Handling call
0709 12:19:38.084651 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:39.011349 (+926698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.538287 (+526938us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400362754. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:39.538572 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129761664}
W20260709 12:19:39.698232 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 122985us
I20260709 12:19:39.700146 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2583) took 1864 ms. Trace:
I20260709 12:19:39.605693 29964 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 12:19:39.746531 25784 rpcz_store.cc:276] 0709 12:19:37.835304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.123978 (+288674us) service_pool.cc:224] Handling call
0709 12:19:38.158331 (+ 34353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:39.272368 (+1114037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:39.272402 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:39.287169 (+ 14767us) log.cc:844] Serialized 26 byte log entry
0709 12:19:39.560073 (+272904us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:39.560120 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.560204 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.699299 (+139095us) raft_consensus.cc:1685] finished
0709 12:19:39.699380 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.699864 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1016406,"raft.queue_time_us":1074899,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":330460,"threads_started":2,"wal-append.queue_time_us":310142}
I20260709 12:19:39.749671 29964 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:39.701956 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2588) took 1773 ms. Trace:
I20260709 12:19:39.750384 29964 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:39.750545 25785 rpcz_store.cc:276] 0709 12:19:37.928276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.002257 (+1073981us) service_pool.cc:224] Handling call
0709 12:19:39.002761 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:39.428683 (+425922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:39.467077 (+ 38394us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:39.467718 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 12:19:39.511211 (+ 43493us) spinlock_profiling.cc:243] Waited 9.96 ms on lock 0x7b4800250158. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29ba8b8aa1 00007f29ba8afb37 00007f29ba8af9ca 00007f29ba9a726f 00007f29ba9a6f37 00007f29ba9bb55c 00007f29ba9bb508 00007f29ba9bb488 00007f29ba9bb443 00007f29ba9ba6ec 00007f29b98e4a96
0709 12:19:39.699739 (+188528us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:39.700560 (+   821us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.701000 (+   440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.701176 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.701238 (+    62us) raft_consensus.cc:1685] finished
0709 12:19:39.701372 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.701690 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276673,"prepare.queue_time_us":34630,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":331143,"raft.run_cpu_time_us":2975,"raft.run_wall_time_us":205520,"spinlock_wait_cycles":22896896,"thread_start_us":300394,"threads_started":2,"wal-append.queue_time_us":136858}
I20260709 12:19:39.752947 29964 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.600632 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:39.702791 26338 raft_consensus.cc:2368] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15.
I20260709 12:19:39.706439 30238 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:39.706900 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.708979 30190 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.710994 30463 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:39.688613 25630 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:39.727528 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2107) took 1591 ms (client timeout 1536 ms). Trace:
I20260709 12:19:39.728718 30482 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:39.730599 25783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 857) took 2406 ms (client timeout 1619 ms). Trace:
I20260709 12:19:39.735742 30758 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.736501 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 865) took 2410 ms. Trace:
I20260709 12:19:39.737587 30762 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:39.701081 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.762038 30758 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.762748 30238 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:39.763767 30238 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:39.764696 30238 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.758553 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:39.758235 25779 rpcz_store.cc:269] 0709 12:19:38.135355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.158429 (+ 23074us) service_pool.cc:224] Handling call
0709 12:19:39.726938 (+1568509us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b5400099f54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:39.727189 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687103360}
I20260709 12:19:39.767143 25784 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:39.766901 30762 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.768085 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.768729 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:39.768976 25779 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:39.769879 30762 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.770275 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.761236 25782 rpcz_store.cc:276] 0709 12:19:37.326222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.529532 (+203310us) service_pool.cc:224] Handling call
0709 12:19:38.494574 (+965042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.748595 (+254021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.735957 (+987362us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54001257d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:39.736263 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218755840}
I20260709 12:19:39.772917 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:39.773084 30566 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:39.759405 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 158794us
I20260709 12:19:39.759526 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.754041 30416 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:39.766199 30758 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.774443 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.762398 30482 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:39.760169 25783 rpcz_store.cc:269] 0709 12:19:37.323995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.382734 (+ 58739us) service_pool.cc:224] Handling call
0709 12:19:39.730028 (+2347294us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540009a454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:39.730329 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4758537088}
I20260709 12:19:39.788416 29987 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.775895 30546 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:39.791109 30567 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:39.774086 30566 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.807971 29962 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.808873 29962 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.812103 29962 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.813045 30716 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.791814 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.815457 25781 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:39.816707 29964 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:39.818471 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.821220 30724 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.820705 25533 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:39.794842 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2584) took 1959 ms. Trace:
I20260709 12:19:39.795679 30658 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:39.796306 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.796540 30238 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.778694 25785 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.801069 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:39.801258 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:39.796306 30758 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:39.802762 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.802604 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:39.803504 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.803776 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:39.804792 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:39.778258 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:39.805945 30482 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:39.806128 25782 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:39.806615 30546 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.771243 25784 raft_consensus.cc:2376] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13. Re-sending same reply.
I20260709 12:19:39.849210 26340 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:39.859997 30758 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.862783 26337 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:39.863395 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:39.866411 26342 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:39.867153 30482 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:39.867479 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 89504us
I20260709 12:19:39.868227 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2204) took 1272 ms. Trace:
I20260709 12:19:39.868033 30482 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.868683 25632 rpcz_store.cc:276] 0709 12:19:38.596016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.876895 (+280879us) service_pool.cc:224] Handling call
0709 12:19:38.904218 (+ 27323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:39.036295 (+132077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:39.036347 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:39.036416 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:39.036474 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.867702 (+831228us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540023fd54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:39.867766 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.868008 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303015168}
I20260709 12:19:39.871636 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.872583 30709 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:39.875810 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:39.876029 26342 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:39.870007 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:39.883242 25786 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.834959 30566 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:39.893342 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.823223 25780 rpcz_store.cc:276] 0709 12:19:37.835586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.828171 (+992585us) service_pool.cc:224] Handling call
0709 12:19:38.828527 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:39.409308 (+580781us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:39.551885 (+142577us) log.cc:844] Serialized 14 byte log entry
0709 12:19:39.699184 (+147299us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:39.708722 (+  9538us) log.cc:844] Serialized 52 byte log entry
0709 12:19:39.709027 (+   305us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:39.709073 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.709194 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.754546 (+ 45352us) raft_consensus.cc:1685] finished
0709 12:19:39.754567 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.794609 (+ 40042us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":409797,"prepare.queue_time_us":160033,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":3847,"raft.queue_time_us":481917,"raft.run_cpu_time_us":540,"raft.run_wall_time_us":17818,"spinlock_wait_cycles":1920,"thread_start_us":351858,"threads_started":3,"wal-append.queue_time_us":73726}
I20260709 12:19:39.898324 25533 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:39.902055 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.903493 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.913136 26339 raft_consensus.cc:2368] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12.
I20260709 12:19:39.915985 25784 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:39.917588 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:39.914074 25782 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.919935 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:39.925980 30597 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:39.927008 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:39.935227 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:39.935525 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.932102 26340 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:39.939309 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:39.906451 30238 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:19:39.943004 25780 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:39.939877 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:39.948659 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 856) took 2624 ms (client timeout 2761 ms). Trace:
I20260709 12:19:39.948930 25636 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:39.949280 25778 rpcz_store.cc:269] 0709 12:19:37.323748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.524663 (+200915us) service_pool.cc:224] Handling call
0709 12:19:39.948202 (+2423539us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400320854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:39.948445 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2628486912}
I20260709 12:19:39.950750 30238 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:39.951290 30238 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:39.952287 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44629.9us
I20260709 12:19:39.952557 30238 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:39.954843 29987 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:39.955265 25787 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:39.956110 30574 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:39.957327 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2106) took 1822 ms. Trace:
I20260709 12:19:39.957742 30704 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:39.958190 30721 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:39.961263 30721 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:39.961684 30721 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:39.964177 30463 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:39.965005 30463 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:39.966606 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:39.967084 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:39.968022 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:39.968390 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:39.969151 25711 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:39.966025 30702 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:19:39.969622 25704 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:39.970031 30702 consensus_queue.cc:1243] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:39.970741 30702 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:39.971088 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:39.972693 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:39.973605 25631 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.973738 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:39.980294 25786 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:39.980371 25785 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:39.984721 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:39.985260 25779 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:39.954336 30238 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:39.935530 30546 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:39.957762 25787 rpcz_store.cc:276] 0709 12:19:38.134595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.090943 (+956348us) service_pool.cc:224] Handling call
0709 12:19:39.954676 (+863733us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:39.955876 (+  1200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.957088 (+  1212us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b5400320850. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:39.957257 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985503616}
I20260709 12:19:39.990463 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:39.994648 25783 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:39.995532 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:39.996680 25705 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:39.982975 30702 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:40.000751 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:40.001425 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.004076 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.004740 30722 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.007028 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2113) took 1864 ms. Trace:
I20260709 12:19:40.007596 25787 rpcz_store.cc:276] 0709 12:19:38.140826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.003634 (+1862808us) service_pool.cc:224] Handling call
0709 12:19:40.004334 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.004779 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.004805 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:40.004853 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:40.004901 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.004974 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.005097 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:40.010689 30482 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.011806 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.012298 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.013145 25708 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:40.013546 30702 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.014277 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.014695 25706 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:19:40.015026 30710 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:40.015270 25708 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:40.015806 30702 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:40.016211 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.016373 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.018893 25780 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:40.019085 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.019578 25708 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:40.020248 25710 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 12:19:40.020350 26338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 865) took 1726 ms (client timeout 1609 ms). Trace:
I20260709 12:19:40.021168 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.021309 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:40.022099 25632 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.022841 30765 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:40.023279 30765 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:40.023838 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.024968 25712 raft_consensus.cc:2410] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:19:40.024391 30765 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.026055 30765 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:40.020792 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.027325 25708 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:40.027666 26340 raft_consensus.cc:2410] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:19:40.029021 25708 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:40.029487 29962 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:40.030293 29962 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.032691 29962 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:40.033272 29962 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.033808 29962 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.035988 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:40.038316 30619 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:40.038801 30619 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:40.015815 30710 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.039925 30619 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.041261 30710 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.041504 30619 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.016110 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.047559 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:40.048626 30597 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:19:40.049077 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 161131us
I20260709 12:19:40.049820 30668 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:40.050513 30668 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.051071 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.052661 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.053187 30668 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.060154 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.062860 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.063827 30721 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.022111 25710 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:40.065678 30721 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:40.067034 30597 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:40.067850 30597 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.068426 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.069458 30452 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:40.070446 30597 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:40.071368 25636 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:40.081257 25533 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 5, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 5 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:19:40.082908 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:40.020823 26338 rpcz_store.cc:269] 0709 12:19:38.293978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.294162 (+   184us) service_pool.cc:224] Handling call
0709 12:19:40.020065 (+1725903us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540036ae54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:40.020241 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3213714176}
I20260709 12:19:40.087245 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.088127 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.028964 30722 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.154736 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2111) took 2016 ms. Trace:
I20260709 12:19:40.155305 25785 rpcz_store.cc:276] 0709 12:19:38.138715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.994820 (+1856105us) service_pool.cc:224] Handling call
0709 12:19:40.154036 (+159216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.154343 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.154370 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:40.154435 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:40.154462 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.154536 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.154649 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:40.156554 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:40.156708 30722 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.157508 25713 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:40.159155 30709 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.034186 25781 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:40.106876 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:40.160708 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59691.1us
I20260709 12:19:40.124980 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:40.119201 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:40.112984 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.113207 25778 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:40.105876 25706 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:40.090430 26335 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:40.112824 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.163489 25779 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:40.124775 26344 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:40.106191 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:40.164620 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.096697 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.166436 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.098594 25782 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:19:40.167874 25780 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:40.168176 25632 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:40.120030 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.164040 25783 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:40.174960 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.099195 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.123027 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:40.095342 30779 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:19:40.197886 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:40.029237 30758 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:40.093681 30746 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.856s	user 0.005s	sys 0.001s
I20260709 12:19:40.118709 30768 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.103518 30708 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:40.199512 26339 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:40.123183 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.201898 25533 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 5, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 5 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:40.116358 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.115367 30463 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:40.204030 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.205261 25708 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:40.167661 30710 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:40.205629 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.213325 26343 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:19:40.214282 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 91453.7us
I20260709 12:19:40.217342 25636 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:40.230651 25782 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:40.228860 25778 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
W20260709 12:19:40.230048 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51873.8us
I20260709 12:19:40.233093 25632 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:40.235003 25781 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:40.208930 30704 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.238142 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.237730 25630 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:40.240653 25784 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:40.159821 30709 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.241745 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:40.243000 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 123908us
I20260709 12:19:40.246658 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.246829 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:40.247807 25709 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.250716 25778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:40.251771 25778 raft_consensus.cc:606] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.252197 25778 raft_consensus.cc:614] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:40.254120 25778 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:40.256069 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.257870 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.259119 25781 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:40.268913 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.270270 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.260087 25712 pending_rounds.cc:85] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 5
I20260709 12:19:40.277694 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:40.278210 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.275333 25603 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:40.283640 25708 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:40.284186 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.285411 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.275025 25712 pending_rounds.cc:107] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:19:40.287570 25712 raft_consensus.cc:2889] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:40.279147 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2112) took 2126 ms. Trace:
I20260709 12:19:40.289497 25779 rpcz_store.cc:276] 0709 12:19:38.140029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.987432 (+1847403us) service_pool.cc:224] Handling call
0709 12:19:40.163277 (+175845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.163929 (+   652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.265897 (+101968us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400099550. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:40.266063 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309460352}
I20260709 12:19:40.283519 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.291538 25635 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:40.292289 25779 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 12:19:40.292438 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.265031 30768 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:40.294824 30768 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:40.295260 30768 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.301651 30425 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:40.306573 30758 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.307318 30758 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.309717 30758 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.264549 30704 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.311599 30717 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 12:19:40.265446 30463 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:40.314005 29962 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.283845 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.302124 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:40.315068 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.277261 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:40.320959 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74308.2us
I20260709 12:19:40.323899 30763 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:40.324476 30763 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.324890 30763 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:40.325492 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 60678.5us
W20260709 12:19:40.325668 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.326929 25638 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:40.278594 30710 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.328835 25631 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:40.329641 30710 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:40.330994 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1003) took 1695 ms. Trace:
I20260709 12:19:40.331409 26341 rpcz_store.cc:276] 0709 12:19:38.635391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.635559 (+   168us) service_pool.cc:224] Handling call
0709 12:19:38.635910 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:38.636318 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.636352 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:38.636816 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:19:38.871432 (+234616us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:38.871472 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.871532 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:39.927255 (+1055723us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:40.313657 (+386402us) raft_consensus.cc:1685] finished
0709 12:19:40.313681 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.330760 (+ 17079us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":98295680,"thread_start_us":234135,"threads_started":1,"wal-append.queue_time_us":600181}
I20260709 12:19:40.333813 30763 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.334898 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:40.335667 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 75795.5us
I20260709 12:19:40.337749 30763 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:40.338001 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 57722.2us
I20260709 12:19:40.350129 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.351336 25676 leader_election.cc:400] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:19:40.351688 25676 leader_election.cc:403] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:19:40.339390 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 76702.5us
I20260709 12:19:40.342033 25707 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:40.342809 30713 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:40.246763 30709 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:40.345813 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.346993 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.313329 30704 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:40.348554 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.330323 30710 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:40.349371 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:40.341961 30668 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:40.351449 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.352182 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.343683 30535 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:40.349058 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2119) took 1009 ms. Trace:
I20260709 12:19:40.355759 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.360688 25783 rpcz_store.cc:276] 0709 12:19:39.340009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.345239 (+1005230us) service_pool.cc:224] Handling call
0709 12:19:40.348422 (+  3183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.348707 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.348733 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:40.348790 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:40.348816 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.348875 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.348977 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:40.360728 25713 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:40.369652 25709 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:40.362375 25631 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:19:40.365177 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:40.364399 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2116) took 1019 ms. Trace:
I20260709 12:19:40.368701 25750 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.375694 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 874) took 1082 ms. Trace:
I20260709 12:19:40.359566 30668 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.376148 25784 rpcz_store.cc:276] 0709 12:19:39.289559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.260349 (+970790us) service_pool.cc:224] Handling call
0709 12:19:40.322542 (+ 62193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.361959 (+ 39417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.362000 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:40.367008 (+  5008us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:40.371739 (+  4731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.371821 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.371934 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:40.376869 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.374782 25779 rpcz_store.cc:276] 0709 12:19:39.339293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.307628 (+968335us) service_pool.cc:224] Handling call
0709 12:19:40.314037 (+  6409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.354737 (+ 40700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.354778 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:40.357935 (+  3157us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:40.358052 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.358188 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.358341 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:40.365727 30238 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:40.381950 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:40.387627 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 85748.5us
I20260709 12:19:40.372107 30717 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:19:40.396358 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.396595 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:40.397439 25783 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:40.374938 30710 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:40.383193 25533 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.381400 30668 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.399997 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.402607 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:40.409051 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45470.7us
I20260709 12:19:40.425635 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 868) took 1166 ms. Trace:
I20260709 12:19:40.425228 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:40.436556 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 85535.4us
I20260709 12:19:40.424645 30567 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:40.445165 25782 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:40.447023 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.447926 25783 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:40.445327 25784 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:40.445461 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.426213 25787 rpcz_store.cc:276] 0709 12:19:39.259394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.017023 (+757629us) service_pool.cc:224] Handling call
0709 12:19:40.017424 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.017769 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:40.018517 (+   748us) log.cc:844] Serialized 14 byte log entry
0709 12:19:40.417950 (+399433us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:40.418024 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:40.422543 (+  4519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.422612 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.425562 (+  2950us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24732,"prepare.queue_time_us":189,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":43,"raft.queue_time_us":33012,"thread_start_us":366061,"threads_started":1}
I20260709 12:19:40.420821 30708 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:40.459699 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:40.424930 30597 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:40.449164 25705 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:40.457170 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.462271 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.463375 30597 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.463903 30597 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.466164 30713 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:40.459419 30596 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:40.468129 30763 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:40.467684 30596 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.472292 26335 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:40.473439 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:40.474365 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.475143 25784 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:19:40.468731 30463 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.452623 30765 raft_consensus.cc:2764] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:40.477008 30463 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.449030 30758 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:40.489449 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.483855 29964 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:40.487941 25629 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:40.489974 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 112752us
I20260709 12:19:40.492077 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:40.474480 30763 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:40.495914 25632 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:40.498761 25782 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:40.505501 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2121) took 1165 ms. Trace:
I20260709 12:19:40.512053 25787 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:40.505900 25783 rpcz_store.cc:276] 0709 12:19:39.340488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.460844 (+1120356us) service_pool.cc:224] Handling call
0709 12:19:40.486962 (+ 26118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.487196 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.488347 (+  1151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:40.494791 (+  6444us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:40.505274 (+ 10483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.505346 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.505448 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:40.503041 30763 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:40.517318 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75871.7us
I20260709 12:19:40.471978 30567 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:40.512959 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 171384us
W20260709 12:19:40.520313 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:40.521347 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 118248us
I20260709 12:19:40.502240 30721 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:40.507354 30713 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:40.530274 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.515328 25779 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:40.532148 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.545993 25533 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 5, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:40.547817 25533 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 7, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 7 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:19:40.549368 25533 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:40.540190 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:40.552706 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:40.553191 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:40.553745 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.550305 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.564400 25779 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:19:40.564924 25779 pending_rounds.cc:85] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 8
I20260709 12:19:40.564977 29964 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:40.565276 25779 pending_rounds.cc:107] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:19:40.565847 25779 raft_consensus.cc:2889] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:40.568997 29964 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:40.559456 30758 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:40.576639 29964 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:40.574731 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48066.7us
I20260709 12:19:40.539716 30721 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.579432 25676 leader_election.cc:400] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12.
I20260709 12:19:40.579856 25676 leader_election.cc:403] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:19:40.534276 30713 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.568931 25533 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:19:40.587291 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60985us
I20260709 12:19:40.525823 30710 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.562795 30574 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 12:19:40.592830 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 225875us
I20260709 12:19:40.595424 26335 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:40.583751 30546 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:40.600066 30721 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.598099 25533 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 7 to 9, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:19:40.603240 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61468.6us
I20260709 12:19:40.565940 30768 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.605541 30483 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:40.615711 30721 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12.
I20260709 12:19:40.617265 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:40.597283 30617 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:40.633693 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.635027 25781 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:40.616786 30785 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:19:40.637310 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:40.639761 30617 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:40.627168 30702 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:40.652431 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:40.629925 30596 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.628423 30463 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:40.658704 25533 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 3 to 5, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 5 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:40.667318 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:40.638356 25520 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 60051e5723b2429582407076a003096c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 10d0c7349085463595aedaf7c071e742 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:40.687855 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:40.688359 25520 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 52981.7us
I20260709 12:19:40.689852 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.655644 30758 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:40.698194 25713 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:40.706244 30238 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:40.706663 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2120) took 1366 ms. Trace:
I20260709 12:19:40.706776 30238 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.707062 25782 rpcz_store.cc:276] 0709 12:19:39.340248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.457937 (+1117689us) service_pool.cc:224] Handling call
0709 12:19:40.482108 (+ 24171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.499151 (+ 17043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.706376 (+207225us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b54000b04d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:40.706597 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37819136}
I20260709 12:19:40.708595 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.709224 25782 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:40.710070 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.710927 30792 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.711772 30482 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.711504 30792 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.712347 30482 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.713853 30792 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:40.716313 30789 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:40.720674 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.721487 25779 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:40.722098 30567 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:40.725322 30787 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.236s	user 0.004s	sys 0.000s
I20260709 12:19:40.727870 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:40.728071 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 873) took 1441 ms. Trace:
I20260709 12:19:40.728571 26340 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.728516 25778 rpcz_store.cc:276] 0709 12:19:39.286293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.255080 (+968787us) service_pool.cc:224] Handling call
0709 12:19:40.255563 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:40.391712 (+136149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.391764 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:40.419303 (+ 27539us) log.cc:844] Serialized 26 byte log entry
0709 12:19:40.594983 (+175680us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:40.595032 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.595159 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:40.727834 (+132675us) raft_consensus.cc:1685] finished
0709 12:19:40.727859 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:40.728007 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102091,"raft.queue_time_us":108501,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":10839,"thread_start_us":125156,"threads_started":1,"wal-append.queue_time_us":237953}
I20260709 12:19:40.731630 30768 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.732136 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.732893 26342 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:40.734781 30758 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.735250 30758 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:40.736034 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.737120 30483 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:40.737679 30483 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:40.735162 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.726104 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 875) took 1431 ms. Trace:
I20260709 12:19:40.735298 26344 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:40.746416 25709 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:40.741204 25781 rpcz_store.cc:276] 0709 12:19:39.295179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.281140 (+985961us) service_pool.cc:224] Handling call
0709 12:19:40.289034 (+  7894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.635452 (+346418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.725863 (+ 90411us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400320354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:40.726034 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785764608}
I20260709 12:19:40.755772 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.762053 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.763922 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.764766 25780 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:40.766932 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:40.770630 30483 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.772029 30482 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.774794 26340 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:40.774436 30617 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.775744 30597 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.780498 25779 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:19:40.781035 25779 pending_rounds.cc:85] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 7
I20260709 12:19:40.695797 30702 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.781203 25787 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:40.763110 26337 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
W20260709 12:19:40.782585 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.782565 25779 pending_rounds.cc:107] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:19:40.783708 25779 raft_consensus.cc:2889] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:40.785475 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.789546 30702 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:40.788560 25778 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
W20260709 12:19:40.804302 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 198056us
I20260709 12:19:40.805384 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:40.810510 29964 raft_consensus.cc:2764] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:19:40.805362 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 207316us
I20260709 12:19:40.828251 25779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:19:40.828998 25779 raft_consensus.cc:606] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.829401 25779 raft_consensus.cc:614] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:40.843743 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:40.853839 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:19:40.857702 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:19:40.859043 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 257713us
I20260709 12:19:40.832834 30238 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.864074 30238 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:40.826647 30546 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:40.874717 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.884541 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.887403 30763 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.889417 25783 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:40.897632 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:40.899216 30546 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.904371 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:40.907961 30792 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.908767 30792 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:40.910732 30800 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:40.912056 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:40.912243 29964 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:40.913492 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:40.913256 29964 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:40.915722 25787 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:40.861958 30482 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:40.918006 29964 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:40.922242 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.932847 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:40.933718 25778 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:19:40.925734 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.935633 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.936818 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:40.935560 25787 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:40.921516 30482 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:40.921804 30768 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:40.947840 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 292301us
W20260709 12:19:40.949620 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.921336 30463 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:40.948652 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 157292us
I20260709 12:19:40.961731 25704 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:40.961010 30482 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:40.930904 30758 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:40.985813 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 170796us
I20260709 12:19:40.989962 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:40.991973 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74584.3us
W20260709 12:19:40.992578 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.996316 29964 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:40.985312 30763 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:40.990903 30668 consensus_meta.cc:287] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.007s
I20260709 12:19:41.010445 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:41.012108 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:41.009775 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:41.014463 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:41.016778 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:41.020707 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:41.021838 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.022228 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:41.023319 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.023592 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:41.024886 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:41.030010 30596 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:41.031653 30763 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:41.033003 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.036561 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:41.037106 30713 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:41.033217 30758 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:41.038921 30709 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:41.039529 30709 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:41.040235 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.040625 30709 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.042649 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.043304 30798 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:41.043604 30763 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:41.044641 30763 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:41.002015 30792 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:41.046774 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.047851 30758 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:41.049686 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.052673 30797 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:41.051343 25636 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:41.053376 30797 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.058248 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:41.066710 25635 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:41.077821 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64288.8us
I20260709 12:19:41.080467 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:41.081315 25778 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:41.089123 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:41.028553 30483 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:41.015540 30668 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.113241 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:41.091140 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 56364.3us
W20260709 12:19:41.094578 30790 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.381s	user 0.002s	sys 0.000s
W20260709 12:19:41.095460 30793 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.445s	user 0.004s	sys 0.002s
W20260709 12:19:41.095709 30805 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:41.091248 30713 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.116593 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.083487 30708 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:41.119585 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.119700 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:41.123554 25638 raft_consensus.cc:2368] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
I20260709 12:19:41.131276 30596 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:41.131877 30596 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:41.132120 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2852) took 1027 ms. Trace:
I20260709 12:19:41.132305 30596 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.132506 25712 rpcz_store.cc:276] 0709 12:19:40.104741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.162413 (+ 57672us) service_pool.cc:224] Handling call
0709 12:19:40.207194 (+ 44781us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:40.288343 (+ 81149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.288382 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:40.689688 (+401306us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b2c000127c8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:19:40.722804 (+ 33116us) log.cc:844] Serialized 26 byte log entry
0709 12:19:40.879158 (+156354us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:40.879213 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.879303 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:41.131779 (+252476us) raft_consensus.cc:1685] finished
0709 12:19:41.131802 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:41.132047 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60527488,"thread_start_us":155808,"threads_started":1,"wal-append.queue_time_us":252220}
I20260709 12:19:41.138306 30483 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.139129 25712 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:19:41.139469 25712 pending_rounds.cc:85] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 8
I20260709 12:19:41.139671 25712 pending_rounds.cc:107] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:19:41.139878 25712 raft_consensus.cc:2889] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:41.140612 30483 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:41.148880 30597 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:41.119196 30708 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:41.151469 30668 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:41.128022 30713 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.152061 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:41.159603 25780 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:41.121737 30758 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:41.160758 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:41.163185 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69024.5us
I20260709 12:19:41.161654 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.166880 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.159540 30708 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:41.164597 30702 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:41.180230 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58469us
I20260709 12:19:41.182085 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2110) took 3044 ms. Trace:
W20260709 12:19:41.184038 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.182641 25786 rpcz_store.cc:276] 0709 12:19:38.137713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.992127 (+1854414us) service_pool.cc:224] Handling call
0709 12:19:40.172310 (+180183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:40.242179 (+ 69869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.242220 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:40.277296 (+ 35076us) log.cc:844] Serialized 26 byte log entry
0709 12:19:40.516834 (+239538us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:40.516895 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.516988 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:41.057076 (+540088us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:41.178339 (+121263us) raft_consensus.cc:1685] finished
0709 12:19:41.178371 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:41.181995 (+  3624us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59002,"raft.queue_time_us":50599,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":19210,"spinlock_wait_cycles":97417472,"thread_start_us":176479,"threads_started":1,"wal-append.queue_time_us":279035}
W20260709 12:19:41.187731 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.188081 30713 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:41.194578 30811 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:19:41.196192 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 103706us
I20260709 12:19:41.198707 30702 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:41.199985 30702 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:41.200497 30702 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:41.171380 30821 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:41.204392 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 671252us
I20260709 12:19:41.205361 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:41.207496 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.207955 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.208238 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.210410 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:41.210911 30668 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:41.201792 30702 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.211320 30668 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:41.211741 30668 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.212152 30702 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:41.168129 30797 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:41.218876 30821 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:41.219436 30821 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:41.220342 25629 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:41.221735 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57591.9us
I20260709 12:19:41.220405 30821 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.223893 30238 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:41.226042 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.227236 30238 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:41.230404 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:41.232455 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.206293 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.233399 30546 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:41.234182 30546 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:41.234400 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:41.239014 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.240008 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:41.240104 30783 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.796s	user 0.004s	sys 0.000s
I20260709 12:19:41.240589 30765 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:41.240248 30596 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.240998 30765 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.241377 30765 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.242113 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2221) took 1168 ms. Trace:
I20260709 12:19:41.242517 25633 rpcz_store.cc:276] 0709 12:19:40.073121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.073335 (+   214us) service_pool.cc:224] Handling call
0709 12:19:40.073745 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:40.205314 (+131569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.205361 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:40.205977 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 12:19:40.493774 (+287797us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:40.493829 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:41.241509 (+747680us) spinlock_profiling.cc:243] Waited 93.1 ms on lock 0x7b54001d6ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:41.241577 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:41.241624 (+    47us) raft_consensus.cc:1685] finished
0709 12:19:41.241638 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:41.241901 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7621,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":18651,"spinlock_wait_cycles":214016384,"thread_start_us":287274,"threads_started":1,"wal-append.queue_time_us":387741}
I20260709 12:19:41.244926 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1120) took 1387 ms. Trace:
I20260709 12:19:41.245299 25634 rpcz_store.cc:276] 0709 12:19:39.857143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.864861 (+  7718us) service_pool.cc:224] Handling call
0709 12:19:39.865475 (+   614us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:39.866343 (+   868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:39.866383 (+    40us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:39.897514 (+ 31131us) log.cc:844] Serialized 78 byte log entry
0709 12:19:40.187956 (+290442us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:40.188013 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:40.188115 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:41.239399 (+1051284us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:41.244490 (+  5091us) raft_consensus.cc:1685] finished
0709 12:19:41.244510 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:41.244734 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1216,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":157,"spinlock_wait_cycles":169160064,"thread_start_us":257125,"threads_started":1,"wal-append.queue_time_us":498011}
I20260709 12:19:41.247030 30596 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:41.222563 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:41.256240 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.257602 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.260900 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:41.261138 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:41.262019 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:41.264050 25713 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
W20260709 12:19:41.265914 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43494.5us
I20260709 12:19:41.265942 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:41.267184 25704 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:41.250072 30765 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.275676 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:41.276754 25707 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:41.259454 30798 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:41.287298 30798 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:41.287859 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:41.288059 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.282021 30802 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:41.282197 30483 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:41.283047 25676 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:41.282923 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:41.261499 30763 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:41.265956 30758 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:41.283259 30801 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:41.282893 30809 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:19:41.256088 30574 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:19:41.226085 30482 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.290258 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:41.288225 30798 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.301779 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.300731 30238 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:19:41.308954 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.276561 30596 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:41.357649 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 78182.7us
I20260709 12:19:41.310818 30797 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:19:41.405166 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 103493us
I20260709 12:19:41.310365 30763 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:41.307650 30802 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:41.286260 30708 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.370854 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:41.413518 25783 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:41.309886 30483 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.311106 30801 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.313699 30710 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:41.296110 30597 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:19:41.382642 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2228) took 1040 ms. Trace:
W20260709 12:19:41.384102 30825 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:19:41.296330 30463 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:41.403792 30482 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:41.404325 30238 consensus_queue.cc:1243] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:19:41.362295 30808 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:41.435828 30808 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:41.404374 25709 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:41.436321 30808 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:41.406095 25750 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.405102 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:41.410171 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 142753us
I20260709 12:19:41.409356 30768 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:41.423306 25603 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.428479 30801 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.404661 30596 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:41.427335 30794 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:41.409446 30797 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.282995 30668 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:41.433713 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 132329us
W20260709 12:19:41.426232 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 123926us
I20260709 12:19:41.424286 25630 rpcz_store.cc:276] 0709 12:19:40.341843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.341995 (+   152us) service_pool.cc:224] Handling call
0709 12:19:41.050843 (+708848us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b5800000c98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:19:41.055999 (+  5156us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:41.056466 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:41.056509 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:41.057078 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 12:19:41.154474 (+ 97396us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:41.154541 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:41.154645 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:41.382245 (+227600us) raft_consensus.cc:1685] finished
0709 12:19:41.382270 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:41.382560 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3655,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6025,"spinlock_wait_cycles":9512192,"thread_start_us":96643,"threads_started":1,"wal-append.queue_time_us":218859}
W20260709 12:19:41.451360 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45332.4us
I20260709 12:19:41.452656 25632 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:41.453991 25601 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.454679 30792 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:41.456528 30713 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:41.457255 30792 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:41.457713 30792 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:41.452354 30482 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:19:41.458621 30792 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.455484 30811 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:41.459893 30792 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:41.461823 30668 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:41.446738 30596 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.451241 30768 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:41.427716 30710 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:41.477934 30710 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:41.466325 25603 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.452702 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.467373 30425 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:41.487696 30425 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.467543 30794 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:41.490314 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:41.457932 30713 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:41.491705 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:41.446604 30797 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:41.492250 30794 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.493808 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:41.494093 25782 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:19:41.495800 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.496305 25782 raft_consensus.cc:1217] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 5.5->[7.6-13.8]   Dedup: 7.6->[11.7-13.8]
I20260709 12:19:41.496810 25782 pending_rounds.cc:85] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 6
I20260709 12:19:41.497043 25782 pending_rounds.cc:107] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:19:41.497041 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:41.497551 25782 raft_consensus.cc:2889] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:41.498744 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.500058 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.501417 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:41.501626 30797 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.454255 30802 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.503429 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.503963 30425 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:41.504673 30463 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:41.504944 30813 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:41.505462 30797 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:41.486868 30709 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:41.490119 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.468318 30763 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.491866 30713 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:41.506263 25750 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.516780 25750 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:41.507025 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:41.485412 30801 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.481057 25704 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:41.513598 26340 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:41.470911 30800 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:41.526605 30800 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:41.517535 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:41.517966 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:41.527390 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:41.527942 25707 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:41.523489 30801 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:41.518738 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:41.509460 30483 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:41.521183 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:41.508683 30546 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:41.523097 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:41.512979 30482 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:41.506650 30535 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:41.520817 29962 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:41.530849 25603 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.506309 30811 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:19:41.530169 30463 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:41.522368 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:41.539402 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:41.556456 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.557029 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.558051 29964 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:41.558225 26335 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:41.537410 30238 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.536926 30713 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.543855 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.533924 30758 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:41.544396 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:41.578742 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 173724us
I20260709 12:19:41.527081 30800 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.553158 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:41.583290 25786 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:41.553587 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.556083 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.559176 26343 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:41.559437 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.584424 25603 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.585006 26341 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:41.543959 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.544435 30668 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:41.549623 30801 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.570695 30597 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:41.550100 30482 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:41.555902 30483 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:41.575625 25780 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:19:41.577316 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 138297us
I20260709 12:19:41.549176 30821 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:41.578270 25751 leader_election.cc:400] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
W20260709 12:19:41.634148 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 56128.6us
I20260709 12:19:41.582027 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:41.582406 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:41.554176 30535 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.598088 30801 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.599664 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:41.603204 30811 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:41.579409 30238 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.591709 30717 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:41.598963 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.613974 25787 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:41.615021 25709 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:41.621086 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:41.632323 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52497.2us
I20260709 12:19:41.580606 30546 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:41.591253 30797 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:41.643143 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:41.622283 30713 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:41.591957 30668 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.619469 30482 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:41.672207 25784 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:41.638622 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.648756 25786 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:41.614674 30765 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:41.655149 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 72787.5us
I20260709 12:19:41.636986 30702 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:41.645094 30535 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:41.677827 25787 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:19:41.676177 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:41.689419 25785 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:41.684607 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:41.685202 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.647136 30811 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:41.669450 30425 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:41.690516 26335 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:41.691087 29964 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:41.636768 30798 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.698678 29964 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:41.678617 30668 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:41.673286 30717 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:41.728286 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.686131 30702 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.700292 30811 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:41.644106 30821 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.734846 29964 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.750869 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:41.745456 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:41.673734 30546 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.754235 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:41.757881 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:41.759121 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:41.762140 26342 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.678121 30797 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.727418 30483 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:41.741722 30702 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.772397 30463 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.772936 30797 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:41.776365 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 92287.8us
I20260709 12:19:41.777484 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:41.739892 30798 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.716020 30617 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:19:41.785501 26339 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:41.706696 30833 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
W20260709 12:19:41.799978 30824 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.349s	user 0.000s	sys 0.003s
I20260709 12:19:41.828724 26335 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:41.831920 26342 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:41.833557 30839 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:41.834122 30839 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:41.834585 30839 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:41.835145 30708 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:41.847091 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:41.864678 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 75704.8us
I20260709 12:19:41.865767 26307 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:41.829047 30765 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.785529 30463 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.895921 25750 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:41.891403 25785 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
W20260709 12:19:41.937140 30808 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.000s
I20260709 12:19:41.937696 30808 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.939968 30809 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:41.896793 30708 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.935187 30483 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.896615 30758 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.787422 30717 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:41.883562 30768 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:41.896445 30597 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.929509 30763 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:41.956701 26344 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
W20260709 12:19:41.957599 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 337149us
W20260709 12:19:42.011606 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 298648us
I20260709 12:19:41.948953 30801 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:42.007221 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 241451us
I20260709 12:19:41.896816 30765 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:42.014315 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:42.022639 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:41.955170 30829 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:19:41.960172 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 265702us
W20260709 12:19:42.014627 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 323468us
I20260709 12:19:42.028673 30463 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.029268 30840 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:42.032677 30596 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.003s
I20260709 12:19:42.033157 30596 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.954629 30708 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.044708 25712 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:41.956110 30768 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:41.955874 30758 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:42.059907 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.958073 30597 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:42.074473 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58240.2us
I20260709 12:19:42.042208 30546 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:42.035125 30829 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:42.139285 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:42.140543 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:42.146864 29962 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:42.147396 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:42.120617 30717 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:42.166747 30238 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.168128 30482 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.170691 30798 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:42.081138 30852 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:42.187583 30857 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:42.188071 30846 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:42.189323 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.194116 30758 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.198346 30708 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:42.137099 30850 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:19:42.188550 30859 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.208235 25707 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:42.119402 30829 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:42.211695 29962 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:42.211833 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 64539.8us
I20260709 12:19:42.275755 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:42.247444 30768 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:42.224198 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:42.234705 30808 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.237426 30840 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:42.238392 30794 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:42.227977 30596 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:42.221704 30800 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
W20260709 12:19:42.283181 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 264055us
I20260709 12:19:42.269263 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:42.274264 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:42.267554 29962 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.291657 29962 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:42.237656 30839 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:42.242458 30535 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.514s	user 0.006s	sys 0.002s
W20260709 12:19:42.268445 30710 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.790s	user 0.006s	sys 0.006s
W20260709 12:19:42.269440 30713 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.000s
W20260709 12:19:42.270946 30668 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.000s
I20260709 12:19:42.255468 30765 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.274256 30846 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.267912 30856 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.273818 30482 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.254576 30821 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.320080 30856 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.302014 30794 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.324651 30794 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.293742 30869 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.325548 30869 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.327814 30869 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.327992 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:42.293830 30800 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.331599 30800 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.331977 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:42.332391 30801 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:42.332883 25705 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:42.333700 30870 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:42.334118 30870 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:42.283015 30802 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:42.263077 30859 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.337137 30859 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.299127 30797 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:42.299646 30710 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.340020 25706 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:42.341411 30710 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.300599 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:42.263140 30597 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.342600 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2872) took 1045 ms. Trace:
W20260709 12:19:42.342741 30702 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.540s	user 0.002s	sys 0.009s
I20260709 12:19:42.342995 25706 rpcz_store.cc:276] 0709 12:19:41.297126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.297292 (+   166us) service_pool.cc:224] Handling call
0709 12:19:41.396484 (+ 99192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:42.340388 (+943904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.342288 (+  1900us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b54003ac854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:42.342531 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2169468160}
I20260709 12:19:42.343183 30702 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.344480 25708 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:42.345449 30702 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.304502 25710 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:19:42.300019 25520 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 60051e5723b2429582407076a003096c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 10d0c7349085463595aedaf7c071e742 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:19:42.346895 25520 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46952.3us
I20260709 12:19:42.273937 30852 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:42.347591 30836 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.652s	user 0.002s	sys 0.000s
I20260709 12:19:42.348359 30852 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.350227 30847 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:42.350628 30847 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:42.301740 30668 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.351409 30546 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.352195 30546 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.353690 30668 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.307370 30846 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.351490 30847 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.309296 30839 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.313160 30873 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:42.315101 30866 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.316301 30862 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:19:42.319468 30838 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.623s	user 0.006s	sys 0.004s
I20260709 12:19:42.266467 30871 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.356798 29964 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:42.364619 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 324142us
I20260709 12:19:42.375689 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:42.369052 25705 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:42.388190 26335 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:42.355985 30847 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:42.391794 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 91306.5us
I20260709 12:19:42.390110 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:42.385545 29964 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.297559 30535 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:42.391966 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 167892us
I20260709 12:19:42.399020 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.400206 25709 raft_consensus.cc:2410] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:42.400936 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.403966 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:42.404517 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:42.411159 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:42.413318 25708 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:42.414012 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.415294 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.419905 25633 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:42.420938 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:42.421994 26343 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:42.427738 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.429415 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:42.367332 30851 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:42.427155 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.396134 30535 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.402266 30856 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.397972 30871 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.450519 30871 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.377110 30866 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.453500 30866 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.455185 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.455883 25786 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:42.456774 30758 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.457324 30758 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.460095 30758 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.461534 25707 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:42.463984 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.464730 25782 raft_consensus.cc:2393] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 8: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:42.465687 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:42.471431 30850 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:42.473060 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.474198 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.474794 25780 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:42.475672 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:42.476315 25780 raft_consensus.cc:2368] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:42.483052 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.483973 25778 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:42.486677 30870 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.488476 30870 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:42.494840 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:42.498423 30837 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.898s	user 0.005s	sys 0.000s
I20260709 12:19:42.442672 25634 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:42.504063 29964 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:42.466233 25782 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:42.402119 30668 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:42.507244 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:42.462271 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:42.508332 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46140.5us
W20260709 12:19:42.510596 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 81251.5us
I20260709 12:19:42.512276 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:42.435230 30851 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.516557 29964 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:42.539198 29964 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:42.539609 25780 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:42.535801 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:42.539682 29964 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:42.417551 30842 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:42.543648 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:42.544232 26343 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:19:42.545773 25787 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:42.547067 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:42.546334 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:42.413062 30862 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:42.558175 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.558090 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.418475 30546 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.553498 25787 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:19:42.556593 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:42.561096 26338 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:42.556823 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.557236 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:42.436889 30597 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.541323 30868 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:42.582468 26343 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:42.506919 30846 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.643307 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:42.638870 29962 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.549883 30668 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.600210 30597 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:42.707582 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 410501us
W20260709 12:19:42.709545 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:42.709713 30797 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.711112 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.711447 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.711128 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:42.712723 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:42.748980 25708 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:42.713809 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.714114 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:42.557807 30702 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:42.541705 30871 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:42.714573 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.714846 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2612) took 1131 ms. Trace:
I20260709 12:19:42.715246 30862 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.716564 25631 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:42.718645 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2880) took 1104 ms. Trace:
I20260709 12:19:42.719274 30765 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.719817 30842 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.586591 25778 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:42.552353 30866 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
I20260709 12:19:42.722522 25704 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:42.731679 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2617) took 1146 ms. Trace:
I20260709 12:19:42.735543 30878 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.748333 29964 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.748271 25633 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
W20260709 12:19:42.713235 30811 consensus_meta.cc:287] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.955s	user 0.004s	sys 0.007s
I20260709 12:19:42.752781 25637 raft_consensus.cc:2368] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:42.762593 25781 rpcz_store.cc:276] 0709 12:19:41.585318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.585515 (+   197us) service_pool.cc:224] Handling call
0709 12:19:41.585825 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:41.638141 (+ 52316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:41.638165 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:41.670605 (+ 32440us) log.cc:844] Serialized 52 byte log entry
0709 12:19:41.890469 (+219864us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:41.949374 (+ 58905us) log.cc:844] Serialized 14 byte log entry
0709 12:19:42.724660 (+775286us) log.cc:844] Serialized 14 byte log entry
0709 12:19:42.731178 (+  6518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.731304 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.731375 (+    71us) raft_consensus.cc:1685] finished
0709 12:19:42.731395 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.731599 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":444895,"prepare.queue_time_us":59731,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":86,"raft.queue_time_us":457745,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":10534,"spinlock_wait_cycles":6646144,"thread_start_us":552122,"threads_started":2,"wal-append.queue_time_us":380751}
I20260709 12:19:42.760370 25784 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:42.723006 30763 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:42.767117 25786 raft_consensus.cc:2393] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:42.770013 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 56011.3us
I20260709 12:19:42.754177 29962 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.734807 30463 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.757069 25713 rpcz_store.cc:276] 0709 12:19:41.613869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.614023 (+   154us) service_pool.cc:224] Handling call
0709 12:19:41.647483 (+ 33460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:42.119482 (+471999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:42.119551 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:42.159929 (+ 40378us) log.cc:844] Serialized 26 byte log entry
0709 12:19:42.367139 (+207210us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:42.367189 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.368663 (+  1474us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.718332 (+349669us) raft_consensus.cc:1685] finished
0709 12:19:42.718359 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.718553 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25983,"raft.queue_time_us":37140,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10360,"thread_start_us":538707,"threads_started":2,"wal-append.queue_time_us":371114}
I20260709 12:19:42.778774 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.785575 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:42.786355 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.753926 25783 rpcz_store.cc:276] 0709 12:19:41.583307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.593891 (+ 10584us) service_pool.cc:224] Handling call
0709 12:19:41.637417 (+ 43526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:41.637781 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:41.637812 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:41.650311 (+ 12499us) log.cc:844] Serialized 26 byte log entry
0709 12:19:41.848409 (+198098us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:41.923934 (+ 75525us) log.cc:844] Serialized 14 byte log entry
0709 12:19:42.354378 (+430444us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b1800280c00. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:42.697652 (+343274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.697760 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.697827 (+    67us) raft_consensus.cc:1685] finished
0709 12:19:42.714601 (+ 16774us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.714771 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2990,"prepare.queue_time_us":75334,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1567,"raft.queue_time_us":432317,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":47,"spinlock_wait_cycles":2690944,"thread_start_us":475702,"threads_started":2,"wal-append.queue_time_us":325010}
I20260709 12:19:42.852113 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.860908 25781 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:42.862741 25636 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:42.869699 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:42.870359 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 95130.9us
I20260709 12:19:42.869699 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.871486 25638 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:42.763777 30535 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:42.853564 30829 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.002s
I20260709 12:19:42.764648 30596 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.896838 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:42.900446 25634 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:42.768148 30879 raft_consensus.cc:2764] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:19:42.751801 30797 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.908413 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:42.905315 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:42.851820 30856 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:19:42.876013 30843 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:19:42.903620 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:42.933261 25787 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:42.901161 30535 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.919183 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:42.919680 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:42.940742 25785 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:42.908252 30882 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:42.765513 30765 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.731325 30868 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.930291 26335 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:42.932271 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:42.933069 25782 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:42.904050 30872 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.929028 30708 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.958250 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:42.776755 30878 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.769449 30842 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.980959 30842 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.789563 30811 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.856058 30463 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.982448 25638 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:42.983749 30463 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:42.932394 30879 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.984125 30811 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.985008 30535 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.983777 30597 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:42.984519 30879 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.965960 30872 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:42.903648 30829 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.909953 30596 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.955544 30882 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:42.997375 25781 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:43.011571 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:43.016503 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.001151 26338 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:43.000969 29964 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:43.020102 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:43.023001 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.024451 26336 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:43.012617 25635 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:19:43.047763 29964 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:43.021090 25784 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:43.048357 29964 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.049134 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:43.049831 25781 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:43.002604 30878 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:42.986727 30708 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.051662 30878 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.052234 30878 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.054431 30878 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:43.005419 30851 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:43.005401 30882 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.002918 30797 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:43.059849 25778 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:42.998217 30596 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:42.998526 30668 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:42.999693 30879 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:43.068327 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 56850.3us
I20260709 12:19:43.070755 30800 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:19:43.072301 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.068617 30864 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:43.075738 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:43.077732 25707 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:43.079113 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:43.080516 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.081257 25705 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:43.082209 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.083395 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:43.085532 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.087488 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.100000 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.101248 26338 raft_consensus.cc:2410] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:19:43.006778 30717 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.801s	user 0.011s	sys 0.000s
I20260709 12:19:43.103124 25601 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:43.108649 25708 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:43.127678 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.128669 26336 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:43.101891 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:43.090376 30847 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.644s	user 0.004s	sys 0.008s
I20260709 12:19:43.019739 30482 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.147377 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:43.149009 29964 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.148324 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:43.135717 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:43.156445 25634 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:19:43.154889 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 249080us
I20260709 12:19:43.150238 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:43.123313 25784 pending_rounds.cc:85] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 6
I20260709 12:19:43.190733 26336 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:43.190868 25784 pending_rounds.cc:107] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:19:43.137681 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.189260 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:43.192665 26338 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:43.227895 30708 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.210172 30821 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:19:43.192193 25784 raft_consensus.cc:2889] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:43.194080 26343 raft_consensus.cc:2410] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:19:43.188375 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 52756.4us
I20260709 12:19:43.200371 26307 leader_election.cc:400] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:43.234133 26307 leader_election.cc:403] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:19:43.211027 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.131938 30797 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:19:43.211020 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:42.986424 30763 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.155635 30851 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:43.240013 30851 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.240510 30869 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:43.131735 30889 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:43.216341 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:43.218828 30885 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
W20260709 12:19:43.219856 30887 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:19:43.126538 30668 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.119s
I20260709 12:19:43.152011 30596 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:43.136332 30882 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:43.245416 26336 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:43.256078 30708 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.006301 30872 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:43.245995 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:43.240554 30851 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:43.260442 29964 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:43.199833 30546 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:43.261885 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 345177us
I20260709 12:19:43.190348 30856 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:19:43.243053 30864 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:43.199250 30814 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:43.247851 30889 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:43.263996 30802 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.009s	sys 0.020s
I20260709 12:19:43.270455 29964 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:43.267199 30836 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:19:43.288177 30846 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:43.288671 30846 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:43.270901 30855 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:43.263317 30842 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:19:43.215602 30482 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.276060 30864 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:43.279640 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 262645us
I20260709 12:19:43.284392 30889 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:43.313345 29964 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.328317 29964 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.270390 30596 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:43.295552 30802 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.009s	sys 0.020s
I20260709 12:19:43.294348 30855 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.293708 30829 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:43.334887 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 76022.1us
I20260709 12:19:43.273882 30708 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.270749 30882 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.328877 29964 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.345036 29964 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:43.335934 25603 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:43.339308 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.348140 25708 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
W20260709 12:19:43.340984 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 145334us
I20260709 12:19:43.302181 30864 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.295035 30821 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:19:43.350059 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.351253 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.358276 25713 raft_consensus.cc:2410] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:19:43.358412 25705 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:43.360167 30482 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:43.337069 30855 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.341005 30802 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:43.364070 30708 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:43.365403 25709 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:43.357532 25710 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:43.366878 30765 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.352137 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.352641 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:43.353590 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.350747 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.368604 25708 raft_consensus.cc:2368] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:43.363528 30802 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.313441 30872 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 12:19:43.371853 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 99868.7us
I20260709 12:19:43.373999 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.375108 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.373415 26308 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:43.380638 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.381328 26338 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:43.395077 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:43.395421 26339 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:43.296730 30846 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.396064 25778 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
W20260709 12:19:43.409303 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 133088us
I20260709 12:19:43.409863 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.402464 26308 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:43.371387 30797 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:43.415349 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:43.419108 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 68517.9us
I20260709 12:19:43.380066 30868 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.426115 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:43.431289 25778 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:43.405736 30856 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:43.438537 30868 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.439146 30868 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.415980 30900 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.413947 30907 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:43.487715 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 65492.6us
I20260709 12:19:43.488113 30868 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:43.491993 30882 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.499119 30896 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:43.433049 30846 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:43.480432 30856 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:43.487762 30899 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:19:43.511170 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:43.559209 25778 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:43.518924 30797 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.522991 30821 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:43.520380 30872 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:43.526772 30668 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:43.534850 26336 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:43.535110 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:43.544785 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 134117us
W20260709 12:19:43.502242 30870 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.013s	user 0.012s	sys 0.000s
W20260709 12:19:43.568455 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 196393us
I20260709 12:19:43.575655 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:43.549381 30907 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:43.576769 25713 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:43.577802 30907 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.578616 30855 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.587029 30911 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:43.587520 30911 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.576056 30878 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:43.587893 30911 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.588145 30878 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.588534 30878 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:43.591667 30869 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:43.595088 30896 raft_consensus.cc:2764] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:43.594846 30879 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:43.600174 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 184942us
I20260709 12:19:43.600528 25750 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:43.604099 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:43.543226 30900 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.606101 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.606365 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.608183 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:43.595736 30821 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:43.613732 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:43.614992 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:43.617206 26335 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 7.
I20260709 12:19:43.615121 30870 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:43.615505 30871 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:43.616112 25750 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:43.633113 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:43.633420 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2629) took 1096 ms. Trace:
I20260709 12:19:43.633793 25780 rpcz_store.cc:276] 0709 12:19:42.537350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.551411 (+ 14061us) service_pool.cc:224] Handling call
0709 12:19:42.551706 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:42.992612 (+440906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:42.992664 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:42.992728 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:42.992778 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.633230 (+640452us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54001ddb54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:43.633269 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.633372 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013340544}
I20260709 12:19:43.634667 30708 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:43.635192 30708 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:43.635139 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:43.635862 25780 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:43.619907 30868 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:43.636744 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:43.636209 30708 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.637529 30868 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:43.638046 30868 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.626638 30765 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:43.639882 30856 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:43.636224 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:43.640974 25779 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
W20260709 12:19:43.641911 30898 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:19:43.640324 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:43.639999 30765 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:43.652068 30765 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:43.629786 25676 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:43.630689 30717 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:43.657616 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:43.658012 26309 leader_election.cc:400] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:43.658340 26309 leader_election.cc:403] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:43.661608 25780 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:43.609547 30843 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:43.610074 30882 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:43.596014 30913 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:19:43.674733 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:43.616794 30821 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:43.677578 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:43.638478 30868 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:43.664312 30879 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:43.668344 30856 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:43.684845 30895 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:43.665230 30852 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:19:43.605638 30910 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:19:43.652933 30765 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.669711 30882 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.664458 30869 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:19:43.716502 30895 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.759841 30895 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:43.731827 25635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2489) took 1327 ms (client timeout 1591 ms). Trace:
W20260709 12:19:43.761083 25635 rpcz_store.cc:269] 0709 12:19:42.403885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.404068 (+   183us) service_pool.cc:224] Handling call
0709 12:19:43.731615 (+1327547us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b5400300cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:43.731732 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809938304}
I20260709 12:19:43.660677 30871 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:19:43.733948 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 900) took 1276 ms. Trace:
I20260709 12:19:43.700551 30856 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:43.707252 30821 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.765578 30879 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:43.765888 25786 rpcz_store.cc:276] 0709 12:19:42.457192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.535115 (+ 77923us) service_pool.cc:224] Handling call
0709 12:19:43.733739 (+1198624us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002b5554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:43.733886 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471686912}
I20260709 12:19:43.766637 30821 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:43.682104 30894 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:19:43.769392 25632 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:43.740317 30878 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.772312 30900 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:43.772792 30878 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:43.681767 30808 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:43.773129 30911 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.755990 30882 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.777477 30889 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:43.780243 25633 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
W20260709 12:19:43.752709 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 177139us
I20260709 12:19:43.752848 25778 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:43.753621 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:43.748646 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:43.678275 30717 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:43.712368 30864 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.778098 30596 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:43.742590 30617 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:43.752271 30814 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:43.779444 30765 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:43.784891 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:43.789171 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 365080us
W20260709 12:19:43.832695 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79187.9us
I20260709 12:19:43.843180 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:43.844710 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.843221 25603 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:43.769696 30856 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.832123 30763 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:43.835404 30597 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:43.766268 30879 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:43.862859 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 78065.6us
I20260709 12:19:43.867924 25603 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:43.866027 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 463626us
I20260709 12:19:43.835177 30617 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:43.893922 26308 leader_election.cc:400] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10.
I20260709 12:19:43.852402 30596 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:43.865432 30535 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:43.901036 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.890815 30814 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:43.903096 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:43.869757 30908 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:19:43.904454 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.911219 30802 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:43.905808 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.902504 25708 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:43.932919 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.933785 25712 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:43.934181 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:43.861222 30883 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.912571 25638 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:43.912810 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.914858 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:43.914767 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:43.916637 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 234033us
I20260709 12:19:43.907963 30535 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.917024 25707 raft_consensus.cc:2410] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:43.947875 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.949440 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.918040 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.931376 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:43.950230 25707 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:43.918594 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.937780 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.910020 30907 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:43.938848 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.940896 25778 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.954303 25704 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:43.905592 25635 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:43.941157 25709 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:43.943960 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:43.958077 25712 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:43.869451 30763 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.910934 30596 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:43.959267 25631 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:43.926081 26309 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.578s (SENT) [suppressed 7 similar messages]
I20260709 12:19:43.960675 25704 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:43.906817 30852 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:19:43.939592 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.932955 30872 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:43.934788 30851 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.000s
I20260709 12:19:43.931448 30911 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:43.963918 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.910171 30910 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:19:43.964038 30870 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:43.986160 30900 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.987679 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.959251 30535 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.958285 30883 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:43.999534 30851 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.013080 25778 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:44.014571 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:43.990316 30668 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:19:43.995244 30597 consensus_queue.cc:1059] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:43.979532 30909 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:43.982967 30907 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.008389 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:44.045841 30870 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.962170 30802 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:44.034852 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.994534 25707 raft_consensus.cc:2410] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:44.043298 25709 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:43.958149 30617 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.058710 30763 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:44.059372 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 167504us
I20260709 12:19:44.111940 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:44.067646 30879 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.005967 30872 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.035650 30889 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:44.035981 30924 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:19:44.087029 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2147) took 1189 ms. Trace:
W20260709 12:19:44.092844 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 166804us
I20260709 12:19:44.027575 30708 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:44.114555 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.115032 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.116901 25787 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:44.118662 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.138718 26338 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:44.122915 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:44.125795 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.140327 25785 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:44.127951 25780 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:44.124574 30904 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.129729 30925 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:44.080927 30900 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.108688 30870 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.132879 25784 rpcz_store.cc:276] 0709 12:19:42.897822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.048746 (+150924us) service_pool.cc:224] Handling call
0709 12:19:43.123077 (+ 74331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:43.261584 (+138507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.261631 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:43.262146 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:19:43.333837 (+ 71691us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:43.334199 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 12:19:43.335776 (+  1577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.335884 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:43.775666 (+439782us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18004b35e0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:44.080177 (+304511us) raft_consensus.cc:1685] finished
0709 12:19:44.086726 (+  6549us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:44.086943 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1373,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2102,"raft.queue_time_us":2342,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":6160,"spinlock_wait_cycles":3538816,"thread_start_us":71337,"threads_started":1,"wal-append.queue_time_us":407240}
I20260709 12:19:44.145277 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.145705 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.140149 25750 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:44.155838 25712 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:44.126196 30847 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:44.164003 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 155715us
I20260709 12:19:44.121964 30763 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.164878 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.129120 30918 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:44.166206 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:44.170403 26335 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:44.170763 25634 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:44.168046 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.169198 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.166687 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:44.138288 30808 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.177667 26338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:44.178246 26338 raft_consensus.cc:606] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.178625 26338 raft_consensus.cc:614] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:44.173625 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.176752 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.145948 30574 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.181237 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:44.180703 25632 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:44.180583 25784 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:44.153618 30889 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:44.176257 30802 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.187270 30889 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.189319 30889 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.190079 25631 raft_consensus.cc:1240] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:44.190850 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.191743 26335 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:44.194934 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.153048 30910 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:44.153061 30911 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:44.198480 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.200358 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.108176 30909 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.200852 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.112293 30596 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.164386 30925 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:44.203620 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 63579.6us
I20260709 12:19:44.169193 30862 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:44.170341 30918 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:44.201063 26339 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:44.148407 30708 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.213518 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.215934 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.217643 25631 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:44.152359 30904 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.174223 30847 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.182376 30763 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.207513 30864 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.223739 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.234112 26335 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:44.191334 30808 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.190811 30574 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.239065 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.236187 26335 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:19:44.211393 30862 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.205072 30911 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.207496 30910 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.210693 30882 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.264429 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.262614 25784 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:44.265439 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.269860 26338 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:44.240767 30929 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:19:44.308797 30868 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.605s	user 0.004s	sys 0.005s
I20260709 12:19:44.309639 30763 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:44.272439 30907 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.273840 30900 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.280474 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.311820 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1025) took 1416 ms. Trace:
I20260709 12:19:44.312274 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.312463 26343 rpcz_store.cc:276] 0709 12:19:42.895808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.903361 (+  7553us) service_pool.cc:224] Handling call
0709 12:19:44.311530 (+1408169us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540058ff54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:44.311735 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597905280}
I20260709 12:19:44.282246 30708 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.282887 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.314838 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.315256 30855 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.315917 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.317028 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:44.283468 30883 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.283622 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.318687 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.319288 25637 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:44.320152 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:44.320829 25637 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:44.321624 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.311846 25631 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:44.323628 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.324615 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:44.325070 25631 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:44.284049 30930 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:19:44.271792 30937 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:44.330709 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.330876 30937 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.331494 26343 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:19:44.330940 26344 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:44.333245 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.333429 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.334204 25636 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:44.335105 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.336217 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:44.336540 30926 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:19:44.336615 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.337639 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:44.338660 30909 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.343936 30904 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.344074 30851 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.346602 30574 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.346911 30883 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:44.347348 30883 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.347666 30883 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:44.334264 30907 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:44.291047 26338 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:44.300487 30925 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:44.299772 30942 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:19:44.350978 30872 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.350903 30708 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:44.301390 30951 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:44.352792 30951 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.353065 30910 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:44.353331 30951 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:44.288128 30847 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.347780 25636 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:44.285831 30953 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:44.309320 30868 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.333526 30937 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.322224 25637 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:44.365160 25635 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:19:44.358944 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 68193.4us
I20260709 12:19:44.373529 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:44.375352 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:44.349262 30895 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.001s
I20260709 12:19:44.382261 30911 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:44.384626 30856 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.000s
I20260709 12:19:44.385123 30856 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.386893 25783 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:44.387877 30882 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.388621 30907 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:44.389096 30907 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.389474 30907 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:44.369004 30953 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:44.390910 30953 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:44.392426 30821 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.625s	user 0.008s	sys 0.004s
I20260709 12:19:44.391856 30811 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 7 to 11, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:44.394650 30900 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:44.402068 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 840) took 1028 ms. Trace:
I20260709 12:19:44.407688 25713 rpcz_store.cc:276] 0709 12:19:43.373482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.373661 (+   179us) service_pool.cc:224] Handling call
0709 12:19:44.401869 (+1028208us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400224cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:44.402003 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464614656}
I20260709 12:19:44.410827 30862 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.356042 30918 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.402954 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:44.404708 25709 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:44.404891 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.415699 30868 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.422641 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.423538 26343 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:19:44.424863 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.425884 26338 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:44.435956 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:44.455180 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:44.463306 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64175.5us
I20260709 12:19:44.468427 26343 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:44.474947 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:44.475416 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:44.400183 30870 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.475584 25780 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:44.519459 25712 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:19:44.476619 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.477931 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:44.478703 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 103486us
I20260709 12:19:44.482085 25677 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:44.483068 25631 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:44.429875 30868 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:44.479005 30708 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.472877 30882 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:44.404995 30895 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.472676 30765 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:44.474086 30922 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
W20260709 12:19:44.525772 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 43796.1us
I20260709 12:19:44.533692 25601 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.539517 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.545610 25780 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:44.594537 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:44.595294 25780 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:44.596752 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.597608 25632 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:44.597714 25711 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:44.539004 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.593122 25704 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:44.600495 25713 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:44.585258 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.588905 25705 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:44.574045 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.588762 25709 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:19:44.570015 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 305042us
I20260709 12:19:44.595039 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:44.605162 25710 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:44.533982 29962 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:19:44.587157 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:44.581661 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.571825 30597 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:19:44.523847 26339 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 7 to 11, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:44.613092 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:44.614441 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 80836.1us
I20260709 12:19:44.596781 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.618686 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.554327 30668 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:44.547020 30794 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:19:44.620050 30668 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:44.621299 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 474096us
I20260709 12:19:44.545687 30574 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:44.623565 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.624217 25780 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:44.624574 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:44.622437 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:44.573837 30856 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.614555 30913 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:19:44.650719 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.680663 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:44.681386 25783 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:44.681120 30951 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.627156 30574 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:44.682761 30904 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:44.682893 30574 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:44.683238 30904 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:44.683753 30904 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.578423 30765 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.686281 30808 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.635164 30859 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:44.687497 30883 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.690377 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.691642 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.692634 30710 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:44.693032 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.693238 30710 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:44.693800 30710 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:44.693888 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.694850 26343 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:44.521943 30870 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.524737 30763 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:44.533181 30846 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:44.696117 30846 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.698194 30846 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.662286 30902 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:19:44.714443 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.714682 30902 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:44.715814 30902 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.717216 30902 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:44.662782 30794 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:44.718842 30794 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.721036 30794 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.623435 30925 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:44.663168 30954 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:44.722443 30951 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.572552 30882 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.721477 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.724726 30882 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.725174 30846 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:44.556656 30868 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.727358 30868 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:44.571382 30708 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.729481 30708 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:44.730227 25779 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:44.667611 30535 consensus_meta.cc:287] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.574s	user 0.004s	sys 0.007s
W20260709 12:19:44.629139 25520 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 60051e5723b2429582407076a003096c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 10d0c7349085463595aedaf7c071e742 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:19:44.731220 25520 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 105191us
I20260709 12:19:44.731173 30535 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.541504 30878 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 12:19:44.669078 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:44.732435 25630 raft_consensus.cc:2368] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17.
W20260709 12:19:44.732709 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 63736.8us
I20260709 12:19:44.733345 30535 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.734948 30883 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.735190 26342 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:44.735940 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.736532 26337 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:44.735977 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.738447 25630 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:44.673928 30961 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:44.674829 30668 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.525132 30909 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:44.739254 25636 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:19:44.739070 30961 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.739745 30866 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:44.740311 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.740468 30871 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:44.741412 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:44.741504 30961 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.741631 30871 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:44.742187 30871 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:44.740331 30866 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:44.743182 30871 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.743798 30866 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.744599 30871 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:44.745400 25781 raft_consensus.cc:3055] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:44.745798 25781 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:44.699426 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.701462 26338 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:44.749006 30971 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:44.751977 25603 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:44.695868 30763 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:44.740043 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.703419 30839 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:44.754262 30909 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:44.759536 30909 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.753545 30763 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.759896 30909 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:44.755239 25786 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:44.761303 30953 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.762280 25786 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:44.762476 25782 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:44.695842 30870 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:19:44.704403 30802 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:44.705058 30843 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:44.708081 30907 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:44.804107 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:44.804895 25706 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.805099 30907 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:44.805846 30907 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:44.806763 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.807653 25635 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:44.708927 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.765168 25781 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.712128 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.722044 30954 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.663689 30952 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:44.738279 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.669288 30895 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.757987 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:19:44.703483 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 81171.7us
I20260709 12:19:44.785281 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:44.786648 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:44.769992 30840 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.003s	sys 0.012s
I20260709 12:19:44.694240 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.773552 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.792172 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 1145) took 2080 ms. Trace:
I20260709 12:19:44.795279 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.797375 25710 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:44.802064 25711 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:44.758220 30971 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:44.758680 30717 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:19:44.757048 30839 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:44.802637 30843 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:44.809185 25781 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:44.786648 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.814124 26338 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:44.815948 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:44.817406 25629 rpcz_store.cc:276] 0709 12:19:42.711550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.714249 (+  2699us) service_pool.cc:224] Handling call
0709 12:19:44.777790 (+2063541us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001dc9d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:44.792102 (+ 14312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2613301120}
I20260709 12:19:44.806277 30907 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:44.819083 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45644.9us
W20260709 12:19:44.817047 30840 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.003s	sys 0.012s
W20260709 12:19:44.821869 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 63966.6us
I20260709 12:19:44.831020 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.835098 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:44.842250 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:44.843134 25710 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:44.846846 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51676.3us
I20260709 12:19:44.820333 25705 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:44.850705 30973 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:44.855533 25710 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:44.856556 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.893692 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.894712 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.895151 29962 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:19:44.895926 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48293.2us
W20260709 12:19:44.896175 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61167.3us
I20260709 12:19:44.896876 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:44.898591 30852 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:44.899765 30855 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:44.900202 30855 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.900575 30855 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:44.902771 26336 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a760df354a14284b004635d45d88a3e"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:44.903358 26336 raft_consensus.cc:606] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.910226 26336 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:44.908437 25781 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:44.916599 25781 pending_rounds.cc:85] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 9
I20260709 12:19:44.916956 25781 pending_rounds.cc:107] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:19:44.917207 25781 raft_consensus.cc:2889] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:44.917502 30878 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:44.918102 30878 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:44.924238 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:44.925570 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 109745us
I20260709 12:19:44.926528 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:44.930641 25785 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:44.825946 30925 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:44.829998 30596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.006s	sys 0.019s
I20260709 12:19:44.928439 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:44.810933 30797 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:19:44.944835 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:44.829480 30839 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:44.921700 30617 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:44.959599 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:44.829730 30961 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:44.982867 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.965793 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.969157 26338 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:44.983801 25781 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:44.984925 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:44.985009 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:44.834103 30895 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:44.981907 26337 raft_consensus.cc:2410] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:44.965368 26335 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:44.892038 30859 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:44.988133 26343 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:44.975335 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:44.940292 30482 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:44.985523 25781 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:44.998436 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:44.999298 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:45.002116 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:45.002445 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:44.959133 30596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.006s	sys 0.019s
I20260709 12:19:45.009352 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:44.989024 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:45.014540 26344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.013s	sys 0.005s
W20260709 12:19:45.257123 26344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.013s	sys 0.005s
I20260709 12:19:45.016930 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:45.254453 25704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.008s	sys 0.009s
I20260709 12:19:45.258237 25785 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:45.246629 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:45.254565 25780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.005s	sys 0.014s
W20260709 12:19:45.259579 25780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.005s	sys 0.014s
W20260709 12:19:45.209496 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:45.254264 25751 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.010s	sys 0.007s
W20260709 12:19:45.261209 25751 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.010s	sys 0.007s
W20260709 12:19:45.261564 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 629928us
I20260709 12:19:45.262991 30843 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:44.976883 30873 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:44.975915 30889 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:45.268455 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 279534us
I20260709 12:19:45.199257 25706 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:19:44.969456 30808 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:45.315162 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48133.3us
I20260709 12:19:45.184582 25712 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
I20260709 12:19:45.270696 25781 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
W20260709 12:19:45.258433 25704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.009s	sys 0.009s
I20260709 12:19:44.995805 30895 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:44.995275 30971 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:19:45.036406 30972 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:19:45.338186 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:44.995509 30961 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:45.254691 30869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.007s
W20260709 12:19:45.200707 30955 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 12:19:45.376976 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.344647 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:45.005741 30870 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:45.052980 30897 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:45.040575 30953 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:45.017549 30868 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:45.018257 30856 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:45.042721 30802 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.178654 30811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:56416 (request call id 8) took 5485 ms. Trace:
W20260709 12:19:45.385942 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 457476us
I20260709 12:19:45.015910 30925 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:44.979579 30883 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.350s
W20260709 12:19:45.388532 30869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.012s	sys 0.007s
I20260709 12:19:45.420797 30596 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:44.984148 30708 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.385s
I20260709 12:19:45.460783 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:45.236236 30954 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:45.428099 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 83552.5us
I20260709 12:19:45.339249 30873 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:45.091940 30710 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:45.380143 30961 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:45.201951 30855 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:45.443495 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:45.581881 25706 raft_consensus.cc:2368] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:19:45.582895 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:45.583441 25706 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:45.340178 30859 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
W20260709 12:19:45.439642 26342 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.009s	sys 0.021s
W20260709 12:19:45.597285 26342 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.009s	sys 0.021s
W20260709 12:19:45.446132 25786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.012s	sys 0.019s
I20260709 12:19:45.436604 30976 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.377555 30668 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 12:19:45.423704 25785 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:19:45.432536 30897 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.001559 30878 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:45.430315 30482 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:45.314080 30535 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:45.547041 25784 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.022s	sys 0.026s
W20260709 12:19:45.644855 25784 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.022s	sys 0.026s
I20260709 12:19:45.380939 30922 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:19:45.380610 30808 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:45.438625 30909 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.678s	user 0.012s	sys 0.004s
I20260709 12:19:45.581275 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:45.594432 25631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.017s	sys 0.036s
W20260709 12:19:45.684718 25631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.018s	sys 0.036s
I20260709 12:19:45.451329 30802 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:45.598264 25786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.012s	sys 0.019s
I20260709 12:19:45.603394 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:45.167189 30574 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:45.685844 25711 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
W20260709 12:19:45.629732 26338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.007s	sys 0.029s
I20260709 12:19:45.630779 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:45.450773 30868 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:45.542717 30935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.016s	sys 0.032s
I20260709 12:19:45.436671 30958 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.639019 25705 raft_consensus.cc:2410] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:19:45.642293 26340 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:19:45.646056 26307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.008s	sys 0.051s
W20260709 12:19:45.699415 26307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.009s	sys 0.051s
W20260709 12:19:45.699878 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 683314us
I20260709 12:19:45.435544 30870 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:45.628657 30794 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:19:45.418721 30895 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:45.634230 30668 consensus_queue.cc:1243] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:19:45.686695 26338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.007s	sys 0.029s
W20260709 12:19:45.629997 25787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.017s	sys 0.019s
W20260709 12:19:45.719960 25787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.017s	sys 0.019s
I20260709 12:19:45.687606 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:45.693897 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 112720us
I20260709 12:19:45.637929 30897 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:45.581667 30596 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:45.694813 30935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.016s	sys 0.033s
I20260709 12:19:45.705482 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:45.697139 30839 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.000s
I20260709 12:19:45.725454 26343 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:45.726023 30839 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:45.727520 25713 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:45.705772 30981 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:19:45.722991 30963 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.035s	sys 0.051s
W20260709 12:19:45.696008 30904 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.001s	user 0.010s	sys 0.004s
I20260709 12:19:45.721786 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:45.701001 30922 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:45.698462 30878 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:45.714890 25709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.020s	sys 0.037s
W20260709 12:19:45.785171 25709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.021s	sys 0.037s
I20260709 12:19:45.698596 30953 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:45.704839 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:45.684463 30871 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:45.733394 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:45.735853 25706 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:45.860543 30935 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:45.731047 26310 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.003s	sys 0.080s
W20260709 12:19:45.916051 26310 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.004s	sys 0.080s
W20260709 12:19:45.916831 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.737040 26343 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:45.635797 30976 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:45.718822 30866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.010s	sys 0.048s
I20260709 12:19:45.794641 26344 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:19:45.787195 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 51904.1us
I20260709 12:19:45.922581 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:45.923985 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:45.730088 30963 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.035s	sys 0.051s
I20260709 12:19:45.697891 30909 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:45.440320 30811 rpcz_store.cc:276] 0709 12:19:39.681827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.682609 (+   782us) service_pool.cc:224] Handling call
0709 12:19:39.776021 (+ 93412us) log.cc:844] Serialized 501 byte log entry
0709 12:19:39.954085 (+178064us) spinlock_profiling.cc:243] Waited 9.74 ms on lock 0x7b54004b21a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b59066b8 00007f29b59062fa 00007f29b598a965 00007f29b599603c 00007f29b59956b3 00007f29b598d9fa 00007f29ba9554c9 00007f29b3ed1c54 00007f29b3ed1be2 00007f29b3ed1b11
0709 12:19:45.166716 (+5212631us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58602,"raft.run_cpu_time_us":1560,"raft.run_wall_time_us":8507,"spinlock_wait_cycles":22401920}
I20260709 12:19:45.781025 30535 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:45.784058 30922 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:45.816582 30895 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:45.796478 25780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.018s	sys 0.040s
W20260709 12:19:45.895756 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:45.946147 30985 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.495s	user 0.001s	sys 0.000s
I20260709 12:19:45.783663 30904 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:45.947959 30794 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:45.927927 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 194640us
I20260709 12:19:45.926419 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:45.731741 30873 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:45.722303 30925 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:45.936698 30866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.010s	sys 0.048s
W20260709 12:19:45.896947 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 98266.9us
I20260709 12:19:45.929919 30869 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:45.967029 25780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.019s	sys 0.040s
I20260709 12:19:45.920279 25711 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:45.963210 31001 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:19:45.962754 30839 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:45.966128 30535 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:45.966419 30971 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.022s	sys 0.044s
I20260709 12:19:46.014578 25785 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:19:46.017402 25636 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:45.970861 30922 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:45.937650 30871 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.016052 30995 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:45.893754 30868 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:45.975262 30710 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:45.971866 30708 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.012524 30999 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.123204 25710 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:45.949450 30935 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:46.091243 25677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.011s	sys 0.007s
W20260709 12:19:46.177033 25677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.012s	sys 0.009s
I20260709 12:19:46.151973 30870 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:45.984272 30855 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.131832 25785 pending_rounds.cc:85] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 11
I20260709 12:19:46.133958 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 931) took 1555 ms. Trace:
W20260709 12:19:46.139858 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.179595 25785 pending_rounds.cc:107] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:19:46.180245 25785 raft_consensus.cc:2889] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:46.179728 25780 rpcz_store.cc:276] 0709 12:19:44.578564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.624893 (+ 46329us) service_pool.cc:224] Handling call
0709 12:19:44.625184 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.625373 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:45.260293 (+634920us) log.cc:844] Serialized 14 byte log entry
0709 12:19:46.024733 (+764440us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:46.132450 (+107717us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:46.132660 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.133779 (+  1119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.133901 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":561386,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":198,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":1398604,"threads_started":2,"wal-append.queue_time_us":892377}
W20260709 12:19:46.142608 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.065s	sys 0.075s
I20260709 12:19:46.015895 30794 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:46.157437 25783 pending_rounds.cc:85] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 5
I20260709 12:19:46.117740 30996 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:46.136197 30983 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.129s	user 0.007s	sys 0.005s
W20260709 12:19:46.133476 30971 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.022s	sys 0.045s
W20260709 12:19:46.143193 30990 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.180s	user 0.001s	sys 0.004s
I20260709 12:19:46.133131 30866 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:19:46.152104 30998 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.336s	user 0.002s	sys 0.004s
I20260709 12:19:46.188077 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2652) took 1497 ms. Trace:
W20260709 12:19:46.158339 30993 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.142s	user 0.000s	sys 0.005s
I20260709 12:19:46.120719 30869 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.158443 30574 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:46.184110 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.066s	sys 0.075s
I20260709 12:19:46.186020 25783 pending_rounds.cc:107] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:19:46.145566 30802 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:46.184629 30950 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:19:46.207135 25786 rpcz_store.cc:276] 0709 12:19:44.690598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.763950 (+ 73352us) service_pool.cc:224] Handling call
0709 12:19:44.764279 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.764615 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.764644 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:44.796885 (+ 32241us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.685636 (+888751us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:45.685678 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.685753 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.187801 (+502048us) raft_consensus.cc:1685] finished
0709 12:19:46.187828 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.188006 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14042,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":11533,"thread_start_us":742554,"threads_started":1,"wal-append.queue_time_us":992045}
W20260709 12:19:46.208994 25784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.010s	sys 0.006s
I20260709 12:19:46.210003 25783 raft_consensus.cc:2889] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:46.210211 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 872065us
W20260709 12:19:46.241286 25784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.011s	sys 0.006s
I20260709 12:19:46.241571 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.242821 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.161979 30708 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.203353 30971 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:46.188081 30859 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.017s	sys 0.031s
I20260709 12:19:46.184399 30482 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:46.227200 30976 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.231827 30794 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:46.233439 30918 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:46.234862 30617 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.233s	user 0.009s	sys 0.005s
I20260709 12:19:46.178236 30999 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:46.236640 30991 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 12:19:46.238294 30988 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:19:46.174839 30596 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:46.182199 30546 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:19:46.292407 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49648.3us
I20260709 12:19:46.336246 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.337533 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.190610 30535 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.208439 30996 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:46.292047 30859 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.017s	sys 0.032s
I20260709 12:19:46.321437 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2927) took 1468 ms. Trace:
W20260709 12:19:46.235776 30994 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:19:46.355800 25709 rpcz_store.cc:276] 0709 12:19:44.853075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.910939 (+ 57864us) service_pool.cc:224] Handling call
0709 12:19:44.911367 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.952958 (+ 41591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.953004 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:44.953550 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.795975 (+842425us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:45.796034 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.796098 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.321137 (+525039us) raft_consensus.cc:1685] finished
0709 12:19:46.321170 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.321375 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78107,"raft.queue_time_us":63311,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":20124,"thread_start_us":756506,"threads_started":1,"wal-append.queue_time_us":940939}
W20260709 12:19:46.314675 30990 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:19:46.291623 30710 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:46.297585 30995 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.347273 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.317947 30900 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:46.302661 25782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.025s	sys 0.012s
I20260709 12:19:46.371819 30995 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:46.372402 25782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.025s	sys 0.013s
W20260709 12:19:46.356171 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 429820us
W20260709 12:19:46.372825 31008 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:19:46.373457 30878 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.008s
I20260709 12:19:46.375434 30995 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.359117 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1161) took 1812 ms. Trace:
I20260709 12:19:46.324648 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2278) took 1337 ms. Trace:
I20260709 12:19:46.376641 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.376578 25630 rpcz_store.cc:276] 0709 12:19:44.987104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.987294 (+   190us) service_pool.cc:224] Handling call
0709 12:19:44.987767 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:45.028090 (+ 40323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:45.028139 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:45.050389 (+ 22250us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.339820 (+289431us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:45.339865 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.339941 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.324395 (+984454us) raft_consensus.cc:1685] finished
0709 12:19:46.324425 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.324581 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39756,"raft.queue_time_us":40505,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8910,"thread_start_us":245747,"threads_started":1,"wal-append.queue_time_us":635227}
I20260709 12:19:46.330102 30909 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.331372 30904 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.370635 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 12:19:46.363227 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 1165) took 1778 ms. Trace:
I20260709 12:19:46.371251 30871 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:46.378662 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:46.357870 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 380318us
I20260709 12:19:46.377527 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.380029 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:46.380110 30546 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:46.381071 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.381412 30546 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:46.381634 26335 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:46.381819 30546 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:46.382452 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:46.378439 25635 rpcz_store.cc:276] 0709 12:19:44.584647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.584863 (+   216us) service_pool.cc:224] Handling call
0709 12:19:46.362975 (+1778112us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54001c4dd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:46.363158 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472094976}
I20260709 12:19:46.371268 30900 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:46.385370 30900 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.350806 30596 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:46.386354 30546 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.387686 30546 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:46.387677 31012 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:46.388913 31012 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.391039 30917 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.391729 30917 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.394659 31010 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:46.396136 31010 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:46.387771 25631 rpcz_store.cc:276] 0709 12:19:44.547149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.549763 (+  2614us) service_pool.cc:224] Handling call
0709 12:19:44.550186 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.814160 (+263974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.814194 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:44.830526 (+ 16332us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.685199 (+854673us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:46.150726 (+465527us) log.cc:844] Serialized 14 byte log entry
0709 12:19:46.358634 (+207908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.358753 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.358813 (+    60us) raft_consensus.cc:1685] finished
0709 12:19:46.358842 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.359036 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13689,"prepare.queue_time_us":629794,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4303,"raft.queue_time_us":85806,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":13330,"thread_start_us":1591781,"threads_started":4,"wal-append.queue_time_us":1213848}
I20260709 12:19:46.401579 26309 leader_election.cc:400] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:19:46.402019 26309 leader_election.cc:403] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:46.404678 26335 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:19:46.408147 30896 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 8. Current term is 9.
I20260709 12:19:46.409659 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:46.348639 30976 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:46.412026 30896 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:46.412500 30896 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:46.412964 30896 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:46.415313 30902 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.698s	user 0.013s	sys 0.004s
I20260709 12:19:46.415870 30668 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:46.420136 30546 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:46.427737 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:46.429039 30855 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:46.569689 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:46.570575 25633 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:46.429666 25638 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:46.571918 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:46.572448 25638 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:46.430635 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2925) took 1673 ms. Trace:
I20260709 12:19:46.574676 25713 rpcz_store.cc:276] 0709 12:19:44.757437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.811207 (+ 53770us) service_pool.cc:224] Handling call
0709 12:19:44.824538 (+ 13331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.727854 (+903316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.430273 (+702419us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54001bf154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:46.430550 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076254976}
I20260709 12:19:46.432639 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:46.435777 30907 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.435838 30971 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.442194 30710 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.447611 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2919) took 1696 ms. Trace:
I20260709 12:19:46.578545 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:46.448288 25712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2753) took 1610 ms (client timeout 1727 ms). Trace:
I20260709 12:19:46.460485 30896 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:46.463274 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53705.7us
W20260709 12:19:46.580205 25712 rpcz_store.cc:269] 0709 12:19:44.838266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.838585 (+   319us) service_pool.cc:224] Handling call
0709 12:19:46.444111 (+1605526us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400147b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:46.448172 (+  4061us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463482752}
I20260709 12:19:46.580736 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:19:46.464243 30991 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:19:46.317819 30482 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.553849 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:46.575827 25630 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
I20260709 12:19:46.462184 30708 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.588794 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:46.589632 25633 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:46.578924 25708 rpcz_store.cc:276] 0709 12:19:44.750825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.752377 (+  1552us) service_pool.cc:224] Handling call
0709 12:19:44.752678 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.941086 (+188408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.941128 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:44.941170 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:44.941198 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.444442 (+1503244us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400066b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:46.447316 (+  2874us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.447524 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432865536}
I20260709 12:19:46.592370 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1200) took 1792 ms. Trace:
I20260709 12:19:46.517367 31010 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:46.528086 30904 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.592780 26342 rpcz_store.cc:276] 0709 12:19:44.800155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.800295 (+   140us) service_pool.cc:224] Handling call
0709 12:19:44.800667 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.808819 (+  8152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.808849 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:44.812247 (+  3398us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.597832 (+785585us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:45.597901 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.598021 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.592178 (+994157us) raft_consensus.cc:1685] finished
0709 12:19:46.592196 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.592315 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7673,"raft.queue_time_us":8556,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":4735,"thread_start_us":705499,"threads_started":1,"wal-append.queue_time_us":1128372}
W20260709 12:19:46.582971 30897 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.847s	user 0.011s	sys 0.003s
I20260709 12:19:46.604782 25630 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:46.598577 26340 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 910) took 1631 ms (client timeout 1707 ms). Trace:
I20260709 12:19:46.526710 30917 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.596091 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:46.606552 26340 rpcz_store.cc:269] 0709 12:19:44.967404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.967581 (+   177us) service_pool.cc:224] Handling call
0709 12:19:46.598234 (+1630653us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b540055a354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:46.598466 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877418368}
I20260709 12:19:46.601529 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2639) took 2472 ms. Trace:
W20260709 12:19:46.611435 25679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.191s	user 0.006s	sys 0.016s
W20260709 12:19:46.611901 25679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.006s	sys 0.016s
I20260709 12:19:46.613579 26340 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:19:46.607854 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.616339 26342 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
W20260709 12:19:46.617925 31020 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 12:19:46.621435 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:46.622289 25630 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:46.623477 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:46.610847 25778 rpcz_store.cc:276] 0709 12:19:44.128696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.199552 (+ 70856us) service_pool.cc:224] Handling call
0709 12:19:44.199869 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.544029 (+344160us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:44.923052 (+379023us) log.cc:844] Serialized 14 byte log entry
0709 12:19:45.455275 (+532223us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:45.582535 (+127260us) log.cc:844] Serialized 26 byte log entry
0709 12:19:45.582777 (+   242us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:45.582837 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.582945 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.601316 (+1018371us) raft_consensus.cc:1685] finished
0709 12:19:46.601336 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.601470 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":642436,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":291831,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":15701,"thread_start_us":1155348,"threads_started":3,"wal-append.queue_time_us":910895}
I20260709 12:19:46.604465 30907 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:46.636895 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:46.639369 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:46.639987 25778 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:46.643837 25633 raft_consensus.cc:2368] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
W20260709 12:19:46.618595 31019 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.113s	user 0.001s	sys 0.001s
I20260709 12:19:46.577692 30546 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.578567 30855 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.573910 30668 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.659775 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:46.613682 30999 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.664428 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:46.665673 25785 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:19:46.670362 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:46.671140 25778 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:46.633425 30802 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.644308 30797 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.608652 30904 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:46.690642 31018 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.142s	user 0.000s	sys 0.002s
I20260709 12:19:46.696281 30710 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.699879 30855 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:46.700506 30855 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:46.701663 30922 consensus_meta.cc:287] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.002s
I20260709 12:19:46.700987 30855 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.703949 30668 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.606087 31010 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.707496 31010 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:46.707711 30895 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.000s
I20260709 12:19:46.708956 30896 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:46.709491 30953 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.791s	user 0.012s	sys 0.000s
I20260709 12:19:46.709915 30953 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:46.673204 30909 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:19:46.609786 30897 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.712785 26336 raft_consensus.cc:606] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.713227 26336 raft_consensus.cc:614] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:46.714331 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.714823 30897 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.715628 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57550 (request call id 7) took 1806 ms. Trace:
I20260709 12:19:46.716023 26336 rpcz_store.cc:276] 0709 12:19:44.909003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.910066 (+  1063us) service_pool.cc:224] Handling call
0709 12:19:46.715441 (+1805375us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54004a4554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5989853 00007f29ba95760a 00007f29b3edd7d4 00007f29b3edd762 00007f29b3edd691 00007f29b3edd60c 00007f29b3edc8c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:46.715544 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4134039808}
I20260709 12:19:46.717228 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.718319 26341 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:46.720701 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 911) took 1752 ms. Trace:
I20260709 12:19:46.687865 30802 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.721112 26338 rpcz_store.cc:276] 0709 12:19:44.968143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.001908 (+ 33765us) service_pool.cc:224] Handling call
0709 12:19:45.002281 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:45.724745 (+722464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:45.724795 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:45.731364 (+  6569us) log.cc:844] Serialized 52 byte log entry
0709 12:19:46.253104 (+521740us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:46.253144 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.253219 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.720477 (+467258us) raft_consensus.cc:1685] finished
0709 12:19:46.720497 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.720642 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46088,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":4523,"thread_start_us":1110910,"threads_started":2,"wal-append.queue_time_us":554607}
I20260709 12:19:46.722839 30802 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.722966 30765 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.725260 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2276) took 1987 ms. Trace:
I20260709 12:19:46.725694 25636 rpcz_store.cc:276] 0709 12:19:44.737574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.742252 (+  4678us) service_pool.cc:224] Handling call
0709 12:19:44.768121 (+ 25869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.132977 (+1364856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.724990 (+592013us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002ad0d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:46.725191 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872517504}
I20260709 12:19:46.727671 25708 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:46.734263 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:46.734871 25713 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:46.739444 30958 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:46.740087 30958 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:46.741675 25750 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:46.684444 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:46.743849 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 938) took 1903 ms. Trace:
I20260709 12:19:46.744568 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:46.743168 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.749965 26339 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:46.747740 25784 rpcz_store.cc:276] 0709 12:19:44.840360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.841029 (+   669us) service_pool.cc:224] Handling call
0709 12:19:44.944112 (+103083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.944376 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:45.645351 (+700975us) log.cc:844] Serialized 14 byte log entry
0709 12:19:46.242184 (+596833us) log.cc:844] Serialized 14 byte log entry
0709 12:19:46.733968 (+491784us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b180036c580. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b93873c7 00007f29b59999a7 00007f29b59a90bf 00007f29b59a9038
0709 12:19:46.734221 (+   253us) log.cc:844] Serialized 14 byte log entry
0709 12:19:46.734432 (+   211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:46.743425 (+  8993us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:46.743504 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.743588 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:46.743788 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19616,"prepare.queue_time_us":243579,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":91,"raft.queue_time_us":2598,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":54896,"spinlock_wait_cycles":411593600,"thread_start_us":1239288,"threads_started":2,"wal-append.queue_time_us":676278}
W20260709 12:19:46.744740 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60394.7us
I20260709 12:19:46.744450 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:46.763013 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 201504us
I20260709 12:19:46.769186 25709 raft_consensus.cc:1240] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:46.770030 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:46.770746 25786 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:46.772507 25706 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:19:46.773847 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:46.775494 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:46.776235 25706 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:46.777325 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:46.770252 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 189593us
I20260709 12:19:46.781504 25638 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 8: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:46.782346 25706 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:46.782997 25778 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:46.784464 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.786036 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:46.783376 25750 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:46.748277 31005 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.718s	user 0.000s	sys 0.016s
I20260709 12:19:46.792441 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:46.756937 30856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.052s	sys 0.038s
I20260709 12:19:46.796348 25784 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:46.791765 30896 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:46.702118 30922 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:46.752291 30708 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:46.742475 30958 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:46.803015 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 455831us
I20260709 12:19:46.748405 30954 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:46.806274 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:46.790879 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:46.797806 30856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.244s	user 0.057s	sys 0.038s
I20260709 12:19:46.752528 31027 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:46.791945 30949 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:19:46.790294 30868 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.811589 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:46.838905 25785 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:46.810206 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:46.808465 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 56664.2us
I20260709 12:19:46.812757 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.814718 25778 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:46.789973 31012 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.806635 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:46.828698 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:46.765404 31010 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:19:46.743781 30797 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:46.842710 30668 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:46.838716 30868 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:46.870334 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60220.7us
W20260709 12:19:46.882952 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67379.3us
I20260709 12:19:46.811784 31030 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:46.834251 26308 connection.cc:256] RPC call timeout handler was delayed by -1.141032690999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3608s for a response
I20260709 12:19:46.841382 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:46.833264 31027 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.830248 30856 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:46.833630 30917 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:46.912317 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 129025us
I20260709 12:19:46.909062 25785 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:46.914343 25784 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:46.924521 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:46.923712 30961 consensus_meta.cc:287] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.289s	user 0.015s	sys 0.000s
W20260709 12:19:46.937152 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 95835.8us
I20260709 12:19:46.844388 31015 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:46.894351 30482 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:46.946966 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:46.832733 30954 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.934397 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.946115 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:46.950867 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:46.956228 26339 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:46.948490 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:46.957515 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.958811 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.960510 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:46.916716 30668 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:46.927433 30868 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.961619 25780 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:46.962232 25778 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:46.962553 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:46.940732 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 128061us
I20260709 12:19:46.963047 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.965022 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:46.965456 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.908373 30999 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:46.968662 25780 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:46.958750 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:46.960310 31012 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:46.980628 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:46.986691 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.982837 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.937698 31027 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:46.999363 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:47.000188 26335 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:46.966975 30546 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:46.995047 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:46.968199 30482 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:46.979782 31010 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:46.997442 30918 consensus_meta.cc:287] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.001s
W20260709 12:19:47.052619 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 106848us
I20260709 12:19:46.961874 30961 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:46.999950 30855 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.001632 30896 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:47.058671 26337 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
I20260709 12:19:47.009348 26343 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:47.084419 26343 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:47.009511 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:46.996709 30873 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:46.935761 31030 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.008543 31045 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:47.006841 30925 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:47.091143 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82021.9us
I20260709 12:19:47.054524 30546 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:47.054275 31012 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.048239 30904 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.098961 31045 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:47.100136 31037 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:19:47.099977 30925 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.097151 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:47.086444 30873 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:47.102350 31028 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:19:47.102834 31033 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:19:46.961972 31015 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.106010 25779 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:47.054092 31010 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:47.110608 31039 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:47.131115 31039 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.131511 31039 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:47.132249 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 853) took 1683 ms. Trace:
W20260709 12:19:47.142153 31014 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.814s	user 0.001s	sys 0.002s
I20260709 12:19:47.145201 30974 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:47.145617 30974 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:47.115809 30925 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:47.146471 30974 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.008315 31043 consensus_queue.cc:1059] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 12:19:47.148259 30925 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:47.148432 30974 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:47.148944 31043 raft_consensus.cc:3055] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:47.132763 25704 rpcz_store.cc:276] 0709 12:19:45.449245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.580697 (+131452us) service_pool.cc:224] Handling call
0709 12:19:45.636894 (+ 56197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:45.729264 (+ 92370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:45.729313 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:45.730399 (+  1086us) log.cc:844] Serialized 26 byte log entry
0709 12:19:46.208968 (+478569us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:46.591793 (+382825us) log.cc:844] Serialized 14 byte log entry
0709 12:19:46.592139 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.131844 (+539705us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b54003aa554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:47.131891 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:47.131939 (+    48us) raft_consensus.cc:1685] finished
0709 12:19:47.132022 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:47.132186 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":178068,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":3561,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":7089,"spinlock_wait_cycles":211641088,"thread_start_us":650313,"threads_started":2,"wal-append.queue_time_us":607526}
I20260709 12:19:47.088924 30897 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.120090 31015 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.148840 30925 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.086879 30482 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:47.096668 31044 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:19:47.092590 30668 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.056913 30856 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.129529 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2755) took 2290 ms. Trace:
I20260709 12:19:47.122462 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:47.184267 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61908.5us
W20260709 12:19:47.154211 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.186002 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:47.187004 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.107271 30797 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.118237 30873 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:47.130342 31022 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:19:47.204721 25751 leader_election.cc:400] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17.
I20260709 12:19:47.183640 25710 rpcz_store.cc:276] 0709 12:19:44.839110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.856474 (+ 17364us) service_pool.cc:224] Handling call
0709 12:19:44.856871 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.175729 (+1318858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.129202 (+953473us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002ec2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:47.129426 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709396224}
I20260709 12:19:47.207122 25751 leader_election.cc:403] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:19:47.121505 31007 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.166s	user 0.007s	sys 0.005s
I20260709 12:19:47.209096 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:47.165231 30907 consensus_meta.cc:287] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.001s
I20260709 12:19:47.124410 30596 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:47.216627 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:47.121400 31010 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:47.124330 31011 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:47.159904 30961 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:47.114854 31030 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.124202 31013 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:19:47.268400 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:47.287992 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 353663us
I20260709 12:19:47.288497 25637 raft_consensus.cc:2410] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:19:47.158655 30922 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.177712 30997 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.122738 31046 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:47.149510 31043 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:47.153880 31045 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.213462 30971 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:47.204437 31034 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:47.296051 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 79535.6us
I20260709 12:19:47.213678 30765 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17.
I20260709 12:19:47.287691 30597 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:47.177335 30897 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.304143 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:47.312335 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:47.172832 31027 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.295845 30846 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.322788 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.298026 30668 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:47.291097 31011 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:47.321481 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:47.297236 30808 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:47.325277 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.150331 31012 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:47.324136 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 311022us
I20260709 12:19:47.327129 25634 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:47.311738 30971 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:47.355631 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.366549 25636 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
W20260709 12:19:47.329548 31050 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:19:47.362738 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:47.357643 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:47.325409 31046 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:47.325776 30896 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:47.393550 30896 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:47.394112 30896 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:47.362984 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:47.394948 25638 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:47.396816 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:47.332551 31011 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:47.399333 31012 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:47.333019 30596 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:47.401129 30904 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:47.369560 31023 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:47.365098 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.005s	sys 0.008s
W20260709 12:19:47.402105 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.005s	sys 0.009s
I20260709 12:19:47.286525 31039 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.304770 30997 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.402942 31027 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.365638 31013 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:19:47.404662 30997 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.405074 31013 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:47.405438 31013 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:47.372375 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2659) took 1246 ms. Trace:
I20260709 12:19:47.405889 25781 rpcz_store.cc:276] 0709 12:19:46.125672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.125805 (+   133us) service_pool.cc:224] Handling call
0709 12:19:46.126139 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:46.126571 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.126611 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:46.127125 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:19:46.473147 (+346022us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b1800276c40. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:46.528165 (+ 55018us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:46.780873 (+252708us) log.cc:844] Serialized 14 byte log entry
0709 12:19:46.782117 (+  1244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.782216 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:47.364053 (+581837us) raft_consensus.cc:1685] finished
0709 12:19:47.372065 (+  8012us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:47.372282 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1067527,"prepare.queue_time_us":258036,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":136372,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":855196,"spinlock_wait_cycles":2530048,"thread_start_us":395860,"threads_started":1,"wal-append.queue_time_us":684313}
I20260709 12:19:47.369047 30971 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:47.408350 30935 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.409410 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 941) took 1441 ms. Trace:
I20260709 12:19:47.409761 25779 rpcz_store.cc:276] 0709 12:19:45.968483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.973304 (+  4821us) service_pool.cc:224] Handling call
0709 12:19:45.977181 (+  3877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:47.118891 (+1141710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.409173 (+290282us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400380254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:47.409350 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2595432320}
W20260709 12:19:47.369220 30708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.008s	sys 0.005s
I20260709 12:19:47.372493 31055 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 12:19:47.411700 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 575305us
I20260709 12:19:47.420496 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:47.421137 25710 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:47.412271 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:47.424523 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:47.333245 30846 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.374205 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.382795 26341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1038) took 1007 ms. Trace:
W20260709 12:19:47.428720 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 64117.9us
I20260709 12:19:47.429037 26341 rpcz_store.cc:276] 0709 12:19:46.375403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.375517 (+   114us) service_pool.cc:224] Handling call
0709 12:19:46.375809 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.718769 (+342960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.382539 (+663770us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54004a4554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:47.382729 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":787269888}
I20260709 12:19:47.375284 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.429869 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:47.374040 25751 leader_election.cc:400] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
I20260709 12:19:47.424711 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:47.424784 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:47.433441 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:47.433756 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59829.9us
I20260709 12:19:47.434028 25710 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:19:47.369920 31009 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:47.426460 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:47.414446 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:47.375048 30883 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:47.443241 25704 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:47.414265 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 935) took 2575 ms. Trace:
I20260709 12:19:47.411496 30996 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.437099 26340 raft_consensus.cc:606] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
W20260709 12:19:47.443286 25751 connection.cc:256] RPC call timeout handler was delayed by -1.0490496580000581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
I20260709 12:19:47.438666 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:47.373302 31046 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:47.446447 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:47.330605 30482 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:47.427871 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 503228us
I20260709 12:19:47.442770 30883 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:47.437094 30911 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:47.368456 30868 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.447400 30765 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:47.449649 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:47.479456 25780 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:47.479911 31044 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:47.449649 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:47.481487 25781 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 7.
I20260709 12:19:47.417129 30708 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:47.381398 31058 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:19:47.358982 31043 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.413769 30976 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:47.427738 31026 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:19:47.485083 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:47.444201 25782 rpcz_store.cc:276] 0709 12:19:44.839318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.839451 (+   133us) service_pool.cc:224] Handling call
0709 12:19:44.893105 (+ 53654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.893331 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:45.447173 (+553842us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.390467 (+1943294us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b18002b5dc0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a
0709 12:19:47.413136 (+ 22669us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.413334 (+   198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:47.413391 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:47.413451 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.414020 (+   569us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:47.414199 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":297807,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":90,"raft.queue_time_us":16920,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":8281,"spinlock_wait_cycles":264759936,"thread_start_us":1404623,"threads_started":2,"wal-append.queue_time_us":921685}
I20260709 12:19:47.474922 25786 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:47.462471 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.850966 25638 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:19:47.851176 25636 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 8.
I20260709 12:19:47.464990 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2161) took 2991 ms. Trace:
I20260709 12:19:47.466866 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:47.447781 25709 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:47.450723 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:47.853118 25784 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:47.854318 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:47.451962 25778 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:47.819209 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:47.839488 31043 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:47.833515 31061 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:47.472005 30883 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:47.458645 30897 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:47.465459 31015 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:47.461356 31046 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:47.482447 31039 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.461733 31035 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:47.851483 31045 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:47.827412 30974 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.678s	user 0.017s	sys 0.007s
I20260709 12:19:47.829298 31033 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:47.829463 30971 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:47.830042 31047 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.845s	user 0.001s	sys 0.000s
I20260709 12:19:47.830575 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:47.472153 30911 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.936851 25710 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:47.832718 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:47.758905 26309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.006s	sys 0.037s
W20260709 12:19:47.939900 26309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.007s	sys 0.038s
W20260709 12:19:47.940227 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 953245us
W20260709 12:19:47.745997 31049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.003s	sys 0.019s
I20260709 12:19:47.455358 30925 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:47.482645 31013 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.890611 31015 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:47.545931 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:47.959990 25704 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 8.
W20260709 12:19:47.834916 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 351911us
W20260709 12:19:47.826296 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.474428 30868 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:47.845142 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:47.855813 25781 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:47.984331 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:47.964871 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1168) took 1539 ms. Trace:
I20260709 12:19:47.985500 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:47.986299 25781 raft_consensus.cc:2410] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:47.470796 30840 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:19:47.852152 25787 rpcz_store.cc:276] 0709 12:19:44.473770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.473915 (+   145us) service_pool.cc:224] Handling call
0709 12:19:44.568253 (+ 94338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:44.895973 (+327720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.896003 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:44.896643 (+   640us) log.cc:844] Serialized 52 byte log entry
0709 12:19:45.720382 (+823739us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:45.720437 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:45.720512 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:46.741948 (+1021436us) spinlock_profiling.cc:243] Waited 67.7 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:46.939248 (+197300us) raft_consensus.cc:1685] finished
0709 12:19:46.939271 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:47.464784 (+525513us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b5400099f50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:47.464915 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1629,"raft.queue_time_us":115210,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":96,"spinlock_wait_cycles":371438080,"thread_start_us":1148874,"threads_started":2,"wal-append.queue_time_us":1050982}
I20260709 12:19:47.867695 25782 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:19:47.912237 31025 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.908257 31035 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:47.992233 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:47.904616 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.905243 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.865125 31061 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:47.919726 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:47.868279 30959 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:47.922966 25634 raft_consensus.cc:2410] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:47.865836 30976 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.891355 30918 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:47.945924 26309 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:47.872277 31056 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:47.956579 25710 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:47.963945 31034 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:47.964856 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.022012 25632 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:47.966144 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 947) took 1159 ms. Trace:
I20260709 12:19:47.967325 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2660) took 1830 ms. Trace:
I20260709 12:19:48.022682 25786 rpcz_store.cc:276] 0709 12:19:46.806225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.806340 (+   115us) service_pool.cc:224] Handling call
0709 12:19:47.965936 (+1159596us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b5400320354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:47.966073 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446066944}
I20260709 12:19:48.023972 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:47.968832 25785 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:47.977851 30971 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:47.992547 30839 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.979218 31015 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.942229 30911 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:47.981635 26338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:47.982728 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.985514 25631 rpcz_store.cc:276] 0709 12:19:46.426019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.427973 (+  1954us) service_pool.cc:224] Handling call
0709 12:19:46.428241 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:47.963605 (+1535364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:47.963643 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:47.963686 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:47.963719 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.964650 (+   931us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001d6fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:47.964688 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:47.964813 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3530649984}
I20260709 12:19:48.041087 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.994365 30763 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:47.901870 30597 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:48.030372 31056 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.051219 30597 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.052973 31056 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.053737 30597 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:47.834187 30535 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.055893 30535 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:47.989765 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:48.057075 25636 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:48.057524 25787 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:47.873153 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 406338us
I20260709 12:19:48.058667 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:48.059324 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.059499 31035 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.060003 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.060796 31035 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:47.872531 30935 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.003294 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.004066 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.992102 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:48.004447 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:48.009308 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 96936.2us
I20260709 12:19:47.949003 30866 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:47.971145 26337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1039) took 1594 ms (client timeout 1571 ms). Trace:
W20260709 12:19:47.943499 31049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.003s	sys 0.023s
I20260709 12:19:48.042296 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:47.987484 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:48.042858 31034 consensus_queue.cc:1243] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:48.022873 25783 rpcz_store.cc:276] 0709 12:19:46.137154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.137357 (+   203us) service_pool.cc:224] Handling call
0709 12:19:46.157114 (+ 19757us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:46.751338 (+594224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.751378 (+    40us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:46.785866 (+ 34488us) log.cc:844] Serialized 78 byte log entry
0709 12:19:46.835616 (+ 49750us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:46.877756 (+ 42140us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.369393 (+491637us) log.cc:844] Serialized 14 byte log entry
0709 12:19:47.369649 (+   256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.369740 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:47.829418 (+459678us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:47.967110 (+137692us) raft_consensus.cc:1685] finished
0709 12:19:47.967131 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:47.967271 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":804915,"prepare.queue_time_us":43917,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":100,"raft.queue_time_us":267665,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":842459,"spinlock_wait_cycles":28548224,"thread_start_us":964068,"threads_started":3,"wal-append.queue_time_us":109267}
I20260709 12:19:48.049059 25782 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.038645 31061 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.007582 31025 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:47.936275 31045 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.049934 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:48.030661 31067 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:48.006263 30954 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:48.021930 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 76086.1us
I20260709 12:19:48.062045 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:48.064251 25636 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:48.065034 25635 raft_consensus.cc:2410] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:19:48.068634 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:47.944060 31033 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:48.065603 26337 rpcz_store.cc:269] 0709 12:19:46.376219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.376367 (+   148us) service_pool.cc:224] Handling call
0709 12:19:47.970959 (+1594592us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540044b454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:47.971063 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566801664}
I20260709 12:19:48.135892 25783 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:48.137653 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 75688us
I20260709 12:19:48.093735 25782 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:19:48.139380 31066 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:48.140236 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.143121 25786 raft_consensus.cc:2410] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:19:48.148125 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:48.074048 31010 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.775s	user 0.013s	sys 0.009s
I20260709 12:19:48.064458 30717 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:48.150244 30717 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:48.150230 31010 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.151476 30717 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.152436 30717 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.152725 31010 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.142168 31033 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.062572 30935 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.065481 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:48.086927 30839 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.154955 26338 raft_consensus.cc:606] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.155324 26338 raft_consensus.cc:614] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:48.155393 31060 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:48.071682 30896 consensus_meta.cc:287] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.677s	user 0.008s	sys 0.007s
I20260709 12:19:48.155954 31060 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:48.156450 31060 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:48.156158 30896 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.156612 30839 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.065403 30866 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:48.158193 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 92784.3us
I20260709 12:19:48.158748 30866 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.159502 31023 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:48.083485 25633 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:19:48.148943 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.150259 25635 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 8.
W20260709 12:19:48.168632 25637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2515) took 3212 ms (client timeout 2120 ms). Trace:
W20260709 12:19:48.169054 25637 rpcz_store.cc:269] 0709 12:19:44.956000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.958050 (+  2050us) service_pool.cc:224] Handling call
0709 12:19:48.168423 (+3210373us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54001c5cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:48.168540 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4655563008}
I20260709 12:19:48.169059 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.170828 25635 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:48.063547 30963 consensus_queue.cc:1059] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:19:48.076098 31061 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:48.063395 30918 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:48.063678 31039 raft_consensus.cc:993] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:19:48.149333 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.080681 31025 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:48.075518 30859 consensus_queue.cc:1175] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "955233c139344131a847044f17f8d348" responder_term: 14 status { last_received { term: 13 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:48.113371 25787 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:48.224977 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:48.225736 25787 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:19:48.159613 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:48.076251 30668 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.742s	user 0.018s	sys 0.004s
W20260709 12:19:48.180821 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.065995 25631 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:48.066233 25781 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:48.070938 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2754) took 3232 ms. Trace:
I20260709 12:19:48.211871 31056 raft_consensus.cc:993] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:19:47.998320 30708 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.213208 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.230901 25636 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.214224 25632 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:48.075462 30856 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:48.221648 25637 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:48.081846 30535 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:48.075186 31011 consensus_meta.cc:287] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.003s
I20260709 12:19:48.221968 30917 raft_consensus.cc:1081] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:19:48.081126 31013 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:48.226969 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 67548.2us
W20260709 12:19:48.252131 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.159994 31023 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:48.231823 25636 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:48.256814 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:48.258108 31076 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:19:48.258231 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.258667 31064 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:48.232383 25705 rpcz_store.cc:276] 0709 12:19:44.838802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.856205 (+ 17403us) service_pool.cc:224] Handling call
0709 12:19:48.070717 (+3214512us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b5400203354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:48.070850 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796647296}
I20260709 12:19:48.249552 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.213714 30918 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.243405 30911 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.252193 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.281221 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.086450 31045 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.230346 31025 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:48.082651 30763 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.234417 25603 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:48.263851 25783 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:48.281883 25786 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:48.286427 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.288990 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.289621 25710 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 7.
I20260709 12:19:48.247085 30839 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:48.296006 31046 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.299054 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:48.259143 31064 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.264726 31056 raft_consensus.cc:1081] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:19:48.311813 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.305622 25781 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:48.291744 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:48.253974 31074 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:48.309682 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75343.2us
I20260709 12:19:48.280673 30918 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:48.314798 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:48.315656 25781 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:48.312527 25783 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:48.315045 25603 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.285585 30866 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:19:48.318543 30668 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:48.318992 30668 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.319087 30866 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:48.319800 30866 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:48.320978 30763 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.320729 30866 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.319340 30668 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:48.259598 31034 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:19:48.326414 31023 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.327312 30935 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.300223 30839 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.251140 30856 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.327883 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.331806 25705 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:48.300837 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:48.313035 30765 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.838s	user 0.016s	sys 0.000s
I20260709 12:19:48.298218 31046 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.322250 30866 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:48.256001 31011 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.310330 31064 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:48.330787 30839 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:48.344229 25786 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:48.361088 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.361742 25786 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:48.314049 30535 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:48.357581 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.363786 25783 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:48.344278 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.364861 25704 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:48.366623 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.368124 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:48.368462 25781 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:48.369199 26342 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:48.374853 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:48.374332 25781 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 14. Ops: []
I20260709 12:19:48.375546 25710 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:48.377338 25781 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:48.336829 31025 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:48.379879 25603 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.384059 25780 raft_consensus.cc:1240] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:48.386327 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.386528 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.387219 25783 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:48.387287 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.387578 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.328655 31023 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:48.371694 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 312937us
I20260709 12:19:48.391822 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.393661 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.395496 25706 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:48.360873 30765 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:48.396013 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.397733 25704 raft_consensus.cc:2410] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:19:48.398463 25783 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:48.401383 25634 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:48.372253 30535 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.406900 25710 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:48.373754 31084 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:48.412177 25709 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:48.409943 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.341295 30904 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:19:48.384471 31025 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.417017 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.386682 31041 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.393999 31045 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:48.421833 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:48.419691 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.330941 31074 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.420336 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.428969 25704 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:48.423774 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.404683 25712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a760df354a14284b004635d45d88a3e"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:50646
I20260709 12:19:48.430415 25712 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:48.430584 25781 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:48.430863 25712 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:48.423913 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.392175 30717 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:48.408670 30535 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:48.440030 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:48.424623 31025 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:48.431983 31041 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.459482 31045 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:48.411619 31058 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.035s
I20260709 12:19:48.466089 25779 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:48.480340 31045 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:48.480911 31058 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.000s	sys 0.037s
I20260709 12:19:48.487692 25704 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:48.489986 30856 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.491564 31041 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.492962 30869 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.418654 31084 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.499224 25712 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.449287 30717 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.493755 30668 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.505798 25712 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.502170 30895 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:48.512923 30846 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.513619 30717 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.517947 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:48.518748 25707 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:48.518748 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.521001 25709 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:48.521422 25704 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:48.522109 31031 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.522464 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2668) took 1577 ms. Trace:
I20260709 12:19:48.522629 31031 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.522972 25785 rpcz_store.cc:276] 0709 12:19:46.944705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.949975 (+  5270us) service_pool.cc:224] Handling call
0709 12:19:47.968555 (+1018580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:48.024787 (+ 56232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.522245 (+497458us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400320350. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:48.522396 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340853120}
I20260709 12:19:48.523387 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.524384 25704 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:48.524796 31031 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.526122 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:48.526688 31075 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:19:48.527014 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.525317 31031 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:48.527915 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2767) took 1066 ms. Trace:
I20260709 12:19:48.528968 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.528437 25713 rpcz_store.cc:276] 0709 12:19:47.461299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.462376 (+  1077us) service_pool.cc:224] Handling call
0709 12:19:47.462764 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:47.463134 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:47.463161 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:47.463571 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 12:19:47.762269 (+298698us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:47.762325 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.762418 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.527686 (+765268us) raft_consensus.cc:1685] finished
0709 12:19:48.527712 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.527855 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4053,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":298297,"threads_started":1,"wal-append.queue_time_us":793129}
I20260709 12:19:48.529920 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2516) took 2169 ms. Trace:
I20260709 12:19:48.530769 31062 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:48.533221 30765 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:19:48.533979 30883 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.636s	user 0.008s	sys 0.000s
I20260709 12:19:48.534463 31046 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:48.534933 31046 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:48.536090 31084 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.535765 31046 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.537241 31046 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:48.538358 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.539259 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.540485 31031 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.542732 31031 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.543886 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:48.544549 25637 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:48.545498 31025 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.548517 31059 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:48.540632 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:48.550532 30959 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.551259 30959 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:48.552052 31039 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:48.552531 31039 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.552870 31039 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:48.554226 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 237991us
I20260709 12:19:48.550447 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.557093 25630 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:48.530344 25629 rpcz_store.cc:276] 0709 12:19:46.366005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.366899 (+   894us) service_pool.cc:224] Handling call
0709 12:19:46.371058 (+  4159us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:46.486994 (+115936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.487045 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:48.055007 (+1567962us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b18000d3d80. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a
0709 12:19:48.056333 (+  1326us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b2c00026ae8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:19:48.056828 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:19:48.383522 (+326694us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:48.410835 (+ 27313us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.419239 (+  8404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.419327 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.529687 (+110360us) raft_consensus.cc:1685] finished
0709 12:19:48.529711 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.529842 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146285,"prepare.queue_time_us":29770,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":140846,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":52577,"spinlock_wait_cycles":93619840,"thread_start_us":326322,"threads_started":1,"wal-append.queue_time_us":445401}
I20260709 12:19:48.555370 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.507638 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:48.564215 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 124257us
I20260709 12:19:48.565042 25676 leader_election.cc:400] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:19:48.565332 25676 leader_election.cc:403] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:19:48.565951 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:48.566922 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.567137 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:48.565971 25637 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:48.567137 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:48.571041 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:48.573096 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.573439 25676 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.501103 30840 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:48.575613 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 68084.1us
I20260709 12:19:48.576644 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.571259 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:48.576220 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 141566us
I20260709 12:19:48.586416 25677 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.587651 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.500445 31052 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:48.591329 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.597681 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:48.569531 30717 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:48.557134 31011 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.572160 30851 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:48.628109 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 47892.5us
I20260709 12:19:48.589291 30870 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:48.575342 31088 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:48.577435 31031 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:48.641369 31058 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:48.642149 30904 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:48.602118 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.577358 31077 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:48.644276 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.645459 26337 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:48.658958 25784 raft_consensus.cc:2410] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:19:48.645845 30846 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:48.589511 31041 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.589344 30922 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:48.646186 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.647303 26344 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:48.647905 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.534425 30883 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.678812 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:48.633126 30911 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.687146 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.687734 25780 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:48.688477 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.653843 26338 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:48.593526 30855 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:48.597709 31084 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:48.600241 31083 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.668309 26339 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:48.681514 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.593206 30856 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:48.644101 30896 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.639324 31061 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:48.645990 30797 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:19:48.688964 25780 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:48.704782 25778 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:48.692435 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.693107 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:48.696817 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.665045 31093 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:19:48.699609 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.656437 31077 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.685096 30904 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:48.694949 31060 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.004s
I20260709 12:19:48.731738 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.678850 30922 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.649212 31031 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:48.698439 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:48.712654 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.594779 30840 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.710214 26337 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:48.704047 26339 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:48.708468 30856 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:48.719959 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 293068us
I20260709 12:19:48.723827 25704 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:48.724520 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:48.725565 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:48.751245 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.694626 31058 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:19:48.705073 31066 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.000s
I20260709 12:19:48.700848 31082 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:48.705214 31025 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.748131 25780 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:48.696718 31084 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.708859 30909 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.748723 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:48.764444 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:48.764997 25711 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:48.754928 25712 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:19:48.685801 31041 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.808812 31013 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:48.809787 31013 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.810688 31041 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.812130 31041 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.812247 31013 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.812623 31041 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.816179 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.817255 30959 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.820123 30959 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.763945 30851 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:48.748951 31060 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.824651 25637 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:48.825568 31060 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.824038 30851 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:48.756209 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.831036 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.832154 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.833128 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.834120 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.761186 25710 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:48.765949 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:48.753890 30856 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:48.762588 31066 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:48.767318 30840 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:48.735049 30535 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.771725 30883 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:48.734253 30896 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:48.772578 30904 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.773062 31064 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.755199 31024 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:48.712100 31061 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.705696 31083 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.716212 31077 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:48.783093 30668 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.752141 31031 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:48.783998 30866 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:48.748770 30922 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:48.786785 31039 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.793939 25711 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:48.794462 30918 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.796325 31068 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:48.763010 31084 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:48.798331 31067 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:48.799275 31055 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:48.799875 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 101512us
I20260709 12:19:48.757495 31058 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:48.800622 31025 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:48.801177 30909 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.803164 30895 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:48.762322 31099 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 12:19:48.808771 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.815769 30839 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:48.973896 26341 raft_consensus.cc:2368] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9.
I20260709 12:19:48.758752 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.708743 30855 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:48.841420 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:48.800810 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:48.844188 30765 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:48.845817 30708 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:48.856316 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:48.859725 26343 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:48.976989 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 135633us
W20260709 12:19:48.977408 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121163us
I20260709 12:19:48.977766 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.978132 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.857144 31062 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:48.869539 25781 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:48.875921 26339 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:48.856244 30896 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:48.854411 31041 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:48.944103 25603 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:48.993886 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 49895.2us
I20260709 12:19:48.951719 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.867564 30870 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:48.968067 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2170) took 1117 ms. Trace:
I20260709 12:19:48.936704 30866 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.957835 25787 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:19:48.937089 31013 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.975437 25780 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:48.844898 31046 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:49.034214 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.033798 25784 rpcz_store.cc:276] 0709 12:19:47.850810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.854052 (+  3242us) service_pool.cc:224] Handling call
0709 12:19:48.956457 (+1102405us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54002b4654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:48.967990 (+ 11533us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848019840}
W20260709 12:19:49.035508 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 83888.1us
W20260709 12:19:49.006551 25750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.656s (SENT) [suppressed 22 similar messages]
I20260709 12:19:48.973546 31067 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:48.983079 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:48.944986 31098 raft_consensus.cc:2764] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:19:48.985883 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:48.940577 30717 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:19:48.993734 25704 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:19:48.994912 31062 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.036564 25787 pending_rounds.cc:85] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 7
I20260709 12:19:49.057637 25787 pending_rounds.cc:107] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:19:49.038992 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:48.950525 31055 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:48.979511 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.040776 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.047025 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:48.980182 30708 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:48.988852 30883 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:48.981364 30959 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:48.990732 31034 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.041137 30794 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:49.088795 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:49.055637 25708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.021s	sys 0.033s
I20260709 12:19:48.996572 30896 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:49.057997 25787 raft_consensus.cc:2889] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:49.095368 30839 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.060469 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.097080 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.098155 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:49.099135 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.099413 31083 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.101231 25629 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:19:49.101640 25629 pending_rounds.cc:85] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 8
I20260709 12:19:49.102352 25629 pending_rounds.cc:107] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:19:49.102655 25629 raft_consensus.cc:2889] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:49.048933 31082 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:19:49.042197 31013 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.061216 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.105257 25632 raft_consensus.cc:2368] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:19:49.105717 31013 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:49.061610 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 473996us
I20260709 12:19:49.062862 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:49.106818 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.068917 31062 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:49.063160 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:49.101666 31067 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.049142 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2178) took 1089 ms. Trace:
I20260709 12:19:49.108803 25785 rpcz_store.cc:276] 0709 12:19:47.960131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.524325 (+564194us) service_pool.cc:224] Handling call
0709 12:19:48.524645 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:48.524874 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:48.573692 (+ 48818us) log.cc:844] Serialized 14 byte log entry
0709 12:19:49.048650 (+474958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:49.048738 (+    88us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:49.048802 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.048893 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:49.049069 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50329,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":60231,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":466140,"threads_started":1}
I20260709 12:19:49.110532 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.111912 25785 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:48.986660 30953 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.113269 25785 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:48.994259 31015 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:49.034936 31093 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:48.985374 31035 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:48.982946 31059 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:19:49.041723 31106 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:49.076048 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:49.070206 30668 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:49.044366 30866 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:49.092976 25708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.021s	sys 0.034s
I20260709 12:19:49.094400 31061 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.073637 30708 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.038594 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.149770 25784 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.118747 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:49.150481 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2174) took 1251 ms. Trace:
I20260709 12:19:49.119455 25677 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:48.997522 31088 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:49.131202 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 406006us
I20260709 12:19:49.128486 25780 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:49.169480 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 2296) took 1030 ms. Trace:
I20260709 12:19:49.169936 25636 rpcz_store.cc:276] 0709 12:19:48.139318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.148502 (+  9184us) service_pool.cc:224] Handling call
0709 12:19:49.149268 (+1000766us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b5400364a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:49.169413 (+ 20145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159770496}
W20260709 12:19:49.163473 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44100.3us
I20260709 12:19:49.143349 26344 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:49.174767 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:48.947116 31068 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:49.120479 31093 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:49.142414 31106 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:49.187403 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:49.123011 31034 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.150939 25782 rpcz_store.cc:276] 0709 12:19:47.899311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.990047 (+ 90736us) service_pool.cc:224] Handling call
0709 12:19:49.150270 (+1160223us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b5400257bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:49.150420 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128935808}
I20260709 12:19:49.157877 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:49.201377 25709 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:19:49.159344 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2171) took 1308 ms. Trace:
I20260709 12:19:49.201902 25778 rpcz_store.cc:276] 0709 12:19:47.851064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.878635 (+ 27571us) service_pool.cc:224] Handling call
0709 12:19:47.879040 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:48.708571 (+829531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.159052 (+450481us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b54002b5054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:49.159267 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1898456832}
I20260709 12:19:49.203235 25778 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:49.075202 31026 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:19:49.183715 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:49.239682 31094 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.735s	user 0.003s	sys 0.000s
I20260709 12:19:49.240731 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:49.241544 26336 raft_consensus.cc:2410] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:49.186748 25780 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:49.142758 31035 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:49.244153 31035 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.245893 30811 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:49.146449 30668 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.246345 30811 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:49.246706 30668 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:49.247313 30811 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.248378 30811 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.251057 31106 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.252440 31106 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:49.253575 31074 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:49.255272 31058 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:49.256042 30976 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.256973 30896 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.258098 26339 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:49.259927 30856 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.192057 25707 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:49.262874 25520 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 60051e5723b2429582407076a003096c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 10d0c7349085463595aedaf7c071e742 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:49.147783 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.263262 30668 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.264253 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 952) took 1101 ms. Trace:
I20260709 12:19:49.274385 26336 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:19:49.204454 31064 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:49.204819 25785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:19:49.300143 25785 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:49.300561 25785 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:49.204963 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:49.205400 31107 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:49.205266 31093 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.298291 31087 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:49.207177 30909 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.113099 31055 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.171375 30870 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.230939 30918 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:49.235900 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:49.323333 25706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.011s	sys 0.046s
I20260709 12:19:49.292534 30896 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:49.264715 25786 rpcz_store.cc:276] 0709 12:19:48.163290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.373857 (+210567us) service_pool.cc:224] Handling call
0709 12:19:49.147616 (+773759us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aeee 000000000036eb39 00007f29ba8de1e0 00007f29ba9a70c4 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:19:49.264170 (+116554us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4914816}
I20260709 12:19:49.191619 30535 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.191110 31059 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:49.129837 30855 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.288435 31034 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:49.290469 30668 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.291405 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2176) took 1377 ms. Trace:
I20260709 12:19:49.173303 30839 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:49.291859 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:49.338708 25781 rpcz_store.cc:276] 0709 12:19:47.914050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.455292 (+541242us) service_pool.cc:224] Handling call
0709 12:19:48.475338 (+ 20046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:48.990709 (+515371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.291064 (+300355us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b5400087ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:49.291319 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905980672}
I20260709 12:19:49.292603 25532 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 10, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 10 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:49.295710 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:49.297389 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:49.297760 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:49.341722 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44060.7us
I20260709 12:19:49.309965 31087 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:49.198064 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:49.349413 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 151437us
I20260709 12:19:49.350340 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2189) took 1097 ms. Trace:
I20260709 12:19:49.301791 26336 pending_rounds.cc:85] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 8
I20260709 12:19:49.299584 31041 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.206799 30935 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:49.196519 25784 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
W20260709 12:19:49.356534 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.262313 30856 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:49.238157 31084 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:49.323851 25706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.011s	sys 0.046s
I20260709 12:19:49.237149 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:49.366385 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.367121 25784 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:49.367415 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:49.368320 25710 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:19:49.222108 31062 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.372538 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:49.373718 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:49.374372 25635 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:49.375356 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:49.378830 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.322196 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:49.380525 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 176219us
I20260709 12:19:49.381624 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:49.330046 25785 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.339519 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:49.391296 25532 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:49.433892 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.337774 25709 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:49.341127 25712 raft_consensus.cc:2368] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9.
I20260709 12:19:49.294061 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:49.344050 31034 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:49.335309 31055 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:49.351449 26336 pending_rounds.cc:107] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:19:49.580605 26336 raft_consensus.cc:2889] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:49.350946 25787 rpcz_store.cc:276] 0709 12:19:48.253343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.813630 (+560287us) service_pool.cc:224] Handling call
0709 12:19:48.957377 (+143747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:49.095374 (+137997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.350045 (+254671us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002b4650. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:49.350244 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329923968}
I20260709 12:19:49.544101 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2954) took 1145 ms. Trace:
I20260709 12:19:49.342764 30839 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:49.546864 25751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.003s	sys 0.016s
W20260709 12:19:49.584169 25751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.003s	sys 0.016s
W20260709 12:19:49.547637 26307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.012s	sys 0.006s
I20260709 12:19:49.550263 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:49.588259 26307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.012s	sys 0.007s
I20260709 12:19:49.588510 25778 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:49.379014 25784 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:49.590852 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.592404 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:49.592972 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.595844 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.596172 25780 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:19:49.596526 31100 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 12:19:49.596858 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.597407 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.600035 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.600994 25780 raft_consensus.cc:2410] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:49.602080 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.604007 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.605281 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.606458 31013 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:49.607168 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.607259 31013 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.610458 31013 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:49.610810 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.611480 25638 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.612134 31084 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:49.612150 25780 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:49.612911 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.613852 25636 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:49.625762 31015 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.506s	user 0.006s	sys 0.001s
I20260709 12:19:49.626688 31087 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.628912 31087 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:49.610703 25778 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:49.632665 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.634320 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:49.588774 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 608537us
I20260709 12:19:49.635465 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:49.638801 25638 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:49.640987 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.352754 31103 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:49.642423 25785 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.643536 25787 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:49.644536 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:49.645129 25787 raft_consensus.cc:2376] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13. Re-sending same reply.
I20260709 12:19:49.645963 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:49.646903 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 73336.6us
I20260709 12:19:49.651151 25780 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:49.297397 30866 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.656301 25780 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:49.665091 25784 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
W20260709 12:19:49.543215 30868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.004s	sys 0.012s
I20260709 12:19:49.714622 25783 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:49.715299 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 80060.9us
I20260709 12:19:49.725824 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:49.368559 31064 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:49.385778 31083 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.621354 30708 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.541708 30802 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:49.383558 30976 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:49.733601 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:49.738646 31024 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:49.739284 31024 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:49.744570 30866 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:49.748332 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 366826us
I20260709 12:19:49.581422 30869 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.572882 31034 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:49.631795 30839 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:49.750427 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:49.380298 30922 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.360829 31041 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.354506 31068 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.820353 25638 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:49.775228 31015 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:49.799526 31067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.017s	sys 0.035s
I20260709 12:19:49.834288 25637 raft_consensus.cc:2368] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:19:49.783968 31034 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:49.829905 25705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.027s	sys 0.035s
W20260709 12:19:49.858956 25705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.028s	sys 0.035s
W20260709 12:19:49.836256 25676 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.022s	sys 0.052s
W20260709 12:19:49.859995 25676 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.022s	sys 0.052s
W20260709 12:19:49.860409 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 692056us
I20260709 12:19:49.752653 31064 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.817615 30535 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:49.762342 31083 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:49.858765 31067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.018s	sys 0.035s
I20260709 12:19:49.879398 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:49.881414 30935 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:49.882073 30935 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:49.885051 30935 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:49.887230 31101 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:19:49.887861 30708 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:49.888779 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2775) took 1365 ms. Trace:
I20260709 12:19:49.889127 25708 rpcz_store.cc:276] 0709 12:19:48.523067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.524938 (+  1871us) service_pool.cc:224] Handling call
0709 12:19:48.525250 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:48.525643 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:48.525671 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:48.526087 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 12:19:49.146912 (+620825us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:49.146951 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.147012 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:49.888544 (+741532us) raft_consensus.cc:1685] finished
0709 12:19:49.888565 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:49.888707 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":620426,"threads_started":1,"wal-append.queue_time_us":806943}
I20260709 12:19:49.890599 31062 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:49.758561 30976 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.894835 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:49.835767 31015 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.784238 30869 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:49.906935 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:49.865917 31031 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.921s	user 0.004s	sys 0.006s
I20260709 12:19:49.910439 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2967) took 1089 ms. Trace:
W20260709 12:19:49.868979 31049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.032s	sys 0.065s
I20260709 12:19:49.912823 25707 rpcz_store.cc:276] 0709 12:19:48.821344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.823447 (+  2103us) service_pool.cc:224] Handling call
0709 12:19:48.823744 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:49.260887 (+437143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.910157 (+649270us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54001bf154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:49.910368 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831171840}
W20260709 12:19:49.922206 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:49.910522 31118 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:19:49.895054 30535 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:49.911826 31117 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 12:19:49.912508 31077 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.967s	user 0.010s	sys 0.000s
W20260709 12:19:49.921098 31049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.033s	sys 0.067s
I20260709 12:19:49.914177 31015 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:49.942880 26309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.017s	sys 0.020s
I20260709 12:19:49.892616 30895 consensus_queue.cc:1059] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 12:19:49.950428 26309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.017s	sys 0.021s
W20260709 12:19:49.954530 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 768643us
W20260709 12:19:49.935788 31035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.003s	sys 0.038s
I20260709 12:19:49.959014 30895 raft_consensus.cc:3055] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:49.971323 30895 raft_consensus.cc:740] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:49.972436 30895 consensus_queue.cc:260] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.973807 30895 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:49.938896 30535 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:49.961190 30922 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:49.956144 30811 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.707s	user 0.008s	sys 0.004s
I20260709 12:19:49.949568 31077 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:49.984550 31077 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:49.987360 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2961) took 1335 ms. Trace:
I20260709 12:19:49.987789 25713 rpcz_store.cc:276] 0709 12:19:48.652092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.652271 (+   179us) service_pool.cc:224] Handling call
0709 12:19:48.652633 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:48.871237 (+218604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:48.871285 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:48.871932 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 12:19:49.171910 (+299978us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:49.171970 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.172065 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:49.987097 (+815032us) raft_consensus.cc:1685] finished
0709 12:19:49.987118 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:49.987295 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150022,"raft.run_cpu_time_us":44,"raft.run_wall_time_us":43,"thread_start_us":517724,"threads_started":2,"wal-append.queue_time_us":855291}
I20260709 12:19:49.994002 25630 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:49.920882 31031 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:50.004282 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.990234 31039 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:49.946465 30904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.007s	sys 0.030s
W20260709 12:19:49.994469 31035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.003s	sys 0.039s
W20260709 12:19:50.006664 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.016271 31034 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.026793 25711 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:50.007920 30904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.008s	sys 0.031s
I20260709 12:19:50.025542 25603 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:50.021728 31093 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.020s	sys 0.041s
W20260709 12:19:50.049173 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.019712 30811 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:50.020200 31077 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:50.023777 30922 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:50.051057 31093 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.020s	sys 0.042s
I20260709 12:19:50.049515 30802 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:50.103493 30976 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:50.028849 31025 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:19:50.098147 31041 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:50.096938 31034 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.122529 31025 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:50.123117 31025 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:50.124104 31025 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.125589 31025 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:50.133595 31083 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:50.082854 30811 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:50.093750 31077 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.117754 30802 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:50.108397 30922 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.139514 31064 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:50.154529 31119 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:19:50.153475 31031 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:50.162159 30959 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:19:50.169306 31087 consensus_meta.cc:287] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.002s
W20260709 12:19:50.167534 31116 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:19:50.174827 30959 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:50.195919 31123 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:19:50.189616 31077 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.198208 31068 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:50.203387 31106 consensus_meta.cc:287] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.951s	user 0.008s	sys 0.008s
W20260709 12:19:50.252218 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 432535us
W20260709 12:19:50.255482 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.265427 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:50.268097 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 390323us
I20260709 12:19:50.282418 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:50.284899 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2536) took 1224 ms. Trace:
I20260709 12:19:50.270881 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2972) took 1448 ms. Trace:
I20260709 12:19:50.290943 25637 rpcz_store.cc:276] 0709 12:19:49.059907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:49.060130 (+   223us) service_pool.cc:224] Handling call
0709 12:19:50.277460 (+1217330us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b540021fa54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:50.284538 (+  7078us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696314240}
I20260709 12:19:50.291059 25705 rpcz_store.cc:276] 0709 12:19:48.822449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.993023 (+170574us) service_pool.cc:224] Handling call
0709 12:19:48.993397 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:49.042372 (+ 48975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:49.042405 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:49.044634 (+  2229us) log.cc:844] Serialized 52 byte log entry
0709 12:19:49.859588 (+814954us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:49.859636 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.859715 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:50.269898 (+410183us) raft_consensus.cc:1685] finished
0709 12:19:50.269957 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:50.270365 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41125,"raft.queue_time_us":39512,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":19575,"thread_start_us":753178,"threads_started":1,"wal-append.queue_time_us":1028125}
I20260709 12:19:50.297973 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:50.300635 26307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.012s	sys 0.057s
W20260709 12:19:50.301194 26307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.012s	sys 0.057s
W20260709 12:19:50.304020 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 570502us
I20260709 12:19:50.308743 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:50.309680 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:50.311661 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1210) took 1504 ms. Trace:
I20260709 12:19:49.582753 25706 rpcz_store.cc:276] 0709 12:19:48.398288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.398928 (+   640us) service_pool.cc:224] Handling call
0709 12:19:48.399175 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:48.399376 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:48.697349 (+297973us) log.cc:844] Serialized 14 byte log entry
0709 12:19:49.543735 (+846386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:49.543793 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:49.543837 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.543911 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:49.544042 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176190,"prepare.queue_time_us":112184,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":178011,"thread_start_us":951660,"threads_started":2}
I20260709 12:19:50.315514 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:50.316939 30866 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:50.319289 31024 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:50.322834 26337 rpcz_store.cc:276] 0709 12:19:48.807241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.817987 (+ 10746us) service_pool.cc:224] Handling call
0709 12:19:48.818272 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:48.818589 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:48.818615 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:48.819049 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:19:49.307024 (+487975us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:49.307085 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.308276 (+  1191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:50.311055 (+1002779us) raft_consensus.cc:1685] finished
0709 12:19:50.311101 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:50.311381 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3826,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":486329,"threads_started":1,"wal-append.queue_time_us":1186229}
I20260709 12:19:50.332077 31027 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.339578 30808 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:50.341463 30808 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.343122 31035 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:50.344058 31035 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:50.333752 31027 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:50.346064 31035 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.357390 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:50.347587 31093 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:19:50.361793 31035 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:50.351322 30870 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:19:50.371227 25704 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:50.358788 26344 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:50.348073 31114 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.387192 31024 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:50.387897 31114 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.386690 26344 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:50.393546 30717 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:50.367789 30802 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.396066 30922 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.353271 31064 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:50.401839 25637 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:50.348912 31055 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:50.377288 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:50.355300 30811 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:50.408499 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.410332 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:50.418877 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:50.420039 25632 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:50.420782 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:50.421093 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:50.422043 25632 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:50.423503 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2968) took 1601 ms. Trace:
I20260709 12:19:50.423944 25711 rpcz_store.cc:276] 0709 12:19:48.821545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.948597 (+127052us) service_pool.cc:224] Handling call
0709 12:19:48.949009 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:50.027378 (+1078369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:50.422921 (+395543us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400105c54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:50.423325 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2478226816}
I20260709 12:19:50.424851 25631 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:50.369184 30856 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:50.428480 25629 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:50.368687 31031 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:19:50.446686 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:50.447440 31031 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:50.448117 31031 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:50.449039 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:50.431103 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:50.434265 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:50.449533 31031 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:50.405645 31064 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:50.186156 30959 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.438189 25632 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:50.355105 31027 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:50.346477 30808 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:50.447661 25711 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.430323 30869 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:50.450407 25712 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:50.435503 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:50.451686 31107 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:50.491755 25752 leader_election.cc:400] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9.
I20260709 12:19:50.495806 25752 leader_election.cc:403] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 12:19:50.452019 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 143552us
I20260709 12:19:50.434027 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:50.458964 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 366520us
W20260709 12:19:50.459856 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 67417.7us
I20260709 12:19:50.462818 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:50.463609 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.465310 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:50.467000 31055 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:50.471385 31114 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.471836 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:50.368172 31066 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:50.452039 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 153908us
W20260709 12:19:50.449215 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 139840us
W20260709 12:19:50.501492 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.507560 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:50.511076 31066 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.514879 25708 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:50.517274 30959 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:50.525009 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:50.525265 30839 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.710s	user 0.004s	sys 0.017s
I20260709 12:19:50.526233 30839 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.527113 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:50.528877 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:50.495039 31107 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:50.530342 31107 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:50.533676 30535 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.006s
I20260709 12:19:50.535153 25705 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:50.488533 31064 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:50.549338 30839 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.536559 26344 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.552738 26344 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:50.558197 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.560348 25712 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:50.563609 25711 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:19:50.499605 30870 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:50.534197 30535 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.514393 31066 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:50.571437 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:50.575322 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 196619us
W20260709 12:19:50.579140 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.590862 25638 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:50.584298 25713 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:50.603435 25705 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
W20260709 12:19:50.586956 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 108545us
I20260709 12:19:50.607148 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:50.584241 31031 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:50.592512 30895 consensus_meta.cc:287] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.618s	user 0.004s	sys 0.020s
I20260709 12:19:50.588275 31084 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.618717 25706 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:50.624236 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:50.623273 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:50.643132 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:50.644093 26337 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:50.645206 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2685) took 1966 ms. Trace:
I20260709 12:19:50.645730 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:50.645805 25783 rpcz_store.cc:276] 0709 12:19:48.650767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.800539 (+149772us) service_pool.cc:224] Handling call
0709 12:19:50.581027 (+1780488us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54000eea54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:50.617356 (+ 36329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614151424}
I20260709 12:19:50.631843 26336 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:50.649117 31084 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.651510 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2779) took 1446 ms. Trace:
I20260709 12:19:50.652035 31084 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:50.642421 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 164576us
I20260709 12:19:50.591199 31034 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 8. Current term is 9.
I20260709 12:19:50.606269 30870 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.651947 25710 rpcz_store.cc:276] 0709 12:19:49.115636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:49.847465 (+731829us) service_pool.cc:224] Handling call
0709 12:19:49.878118 (+ 30653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:49.878351 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:50.270270 (+391919us) log.cc:844] Serialized 14 byte log entry
0709 12:19:50.404454 (+134184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:50.438947 (+ 34493us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:50.486747 (+ 47800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:50.497656 (+ 10909us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:50.562383 (+ 64727us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8178,"prepare.queue_time_us":323953,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":18082,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":23560,"thread_start_us":450091,"threads_started":2,"wal-append.queue_time_us":176254}
I20260709 12:19:50.690868 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:50.693100 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.693017 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:50.694082 26336 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:50.694115 31034 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:50.696079 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 88167.3us
I20260709 12:19:50.696249 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:50.697252 25783 raft_consensus.cc:2393] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:50.697091 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:50.698762 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:50.699628 25786 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:50.702592 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:50.703418 25785 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:50.710048 25786 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:50.711526 31114 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:50.712954 31114 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:50.713342 31114 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:50.714380 31114 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:50.716969 31114 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:50.725373 25783 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:50.728783 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:50.729441 25784 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:50.730969 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:50.731880 26339 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 9: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:50.733259 25785 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:50.735061 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:50.736006 25780 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:50.739617 25711 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:50.741930 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:50.743100 26344 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
W20260709 12:19:50.751799 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.750993 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:50.611651 31055 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:50.752231 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 142652us
I20260709 12:19:50.767961 31151 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:50.768800 31151 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.773414 31151 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.775010 31151 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:50.776662 31151 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:50.780864 31151 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.764833 25784 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
W20260709 12:19:50.786792 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 136737us
I20260709 12:19:50.802912 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:50.790184 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:50.761788 26344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1058) took 1938 ms (client timeout 2200 ms). Trace:
I20260709 12:19:50.763980 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:50.802765 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:50.803617 26344 rpcz_store.cc:269] 0709 12:19:48.815667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:49.204675 (+389008us) service_pool.cc:224] Handling call
0709 12:19:50.358346 (+1153671us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54000a6520. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5906dc0 00007f29b5992e18 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:50.753957 (+395611us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636100864}
I20260709 12:19:50.741732 30895 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:50.809288 31152 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:50.809821 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:50.753801 31066 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.811910 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:50.814633 31059 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
W20260709 12:19:50.816720 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.828725 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:50.818626 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:50.821993 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 107524us
W20260709 12:19:50.825232 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76796.6us
I20260709 12:19:50.824457 25781 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
W20260709 12:19:50.810251 25750 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.656s (SENT)
I20260709 12:19:50.832526 31066 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:50.838269 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:50.839491 25710 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:50.840474 31103 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:50.841102 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.826440 31114 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:50.825682 31152 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.847208 25750 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:50.869398 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:50.869405 25787 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:50.869635 26339 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:50.870324 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:50.871685 25629 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:50.849956 31151 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:50.840687 31066 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:50.888528 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78847.7us
I20260709 12:19:50.852411 30535 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.907441 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:50.847764 30839 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:19:50.873929 31152 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:50.942080 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:50.969733 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:50.931350 31024 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.980085 30870 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:50.908336 30895 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:19:50.989675 26336 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:50.989065 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:50.970715 30808 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.988193 25710 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:19:51.001729 25781 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:51.023828 31164 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:51.024639 31164 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:50.977470 31093 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:19:51.027179 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 171514us
W20260709 12:19:51.003755 31015 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.964s	user 0.019s	sys 0.007s
W20260709 12:19:51.056841 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55339.9us
I20260709 12:19:50.996037 31158 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:51.026677 30808 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:51.069859 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 259638us
I20260709 12:19:51.068768 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:51.071722 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:51.093719 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 215985us
I20260709 12:19:51.016258 31107 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.055823 31093 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.069859 30839 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.110029 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:51.072325 30856 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.121215 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:51.125311 26339 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:51.129379 25629 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:51.129675 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 91970.2us
I20260709 12:19:51.050303 31024 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.140072 25638 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:51.179636 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 367805us
I20260709 12:19:51.188251 26336 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:51.191583 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:51.192956 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.193823 26338 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:51.198514 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 95393us
I20260709 12:19:51.071441 31015 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.134281 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:51.196949 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:51.194450 31099 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:51.231372 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 124363us
W20260709 12:19:51.222849 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78920us
I20260709 12:19:51.141360 31027 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:51.202951 31164 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.140470 31158 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.194604 25532 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.251308 26336 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:51.194798 30839 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.256464 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.236613 30895 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.253252 31150 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:51.251981 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:51.133874 30856 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:51.282153 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:51.333664 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.235762 30811 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:51.289808 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:51.336546 30811 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:51.336994 30811 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:51.337519 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:51.337911 30811 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.338635 30535 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:51.339740 30811 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:51.340889 30535 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:51.341351 30535 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.341751 30535 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:51.347515 31093 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.290583 31161 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:51.359140 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:51.360028 26343 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:51.291292 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:51.291401 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:51.291949 31025 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.138s	user 0.018s	sys 0.003s
I20260709 12:19:51.292531 30808 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.292997 31158 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.284714 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:51.291316 26339 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:19:51.296279 31166 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:51.296622 31165 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:19:51.297551 31155 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:19:51.298377 31123 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:19:51.391852 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 100669us
W20260709 12:19:51.368505 31035 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.993s	user 0.014s	sys 0.001s
I20260709 12:19:51.299994 31015 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.300462 30839 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.301820 31164 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:51.253628 31099 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.302922 31107 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.284336 30895 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.304066 25629 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
W20260709 12:19:51.306515 31031 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.681s	user 0.006s	sys 0.002s
I20260709 12:19:51.306957 26337 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:19:51.408774 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.409713 26336 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:51.284178 31162 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:51.325145 31139 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.849s	user 0.013s	sys 0.003s
I20260709 12:19:51.420346 25710 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:51.288780 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:51.377084 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:51.392854 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:51.390975 25634 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:51.415428 30895 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.416716 25711 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:51.417223 31164 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:51.417615 31025 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:51.411958 31162 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:51.261395 31114 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:51.422062 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:51.442093 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49360.2us
W20260709 12:19:51.444954 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 322995us
W20260709 12:19:51.442834 31064 consensus_meta.cc:287] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.871s	user 0.013s	sys 0.000s
I20260709 12:19:51.452515 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:51.458274 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 223454us
I20260709 12:19:51.459206 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:51.446069 31025 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:51.459995 31025 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:51.442785 31161 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.462884 31161 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.464072 30856 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.465044 31161 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.441569 31165 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.445360 31166 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.471045 31064 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.470923 31114 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.473560 31064 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.450443 31162 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:51.477630 31151 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.564s	user 0.002s	sys 0.008s
I20260709 12:19:51.447773 31093 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:51.478354 31093 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.478760 31093 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:51.482558 26335 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
W20260709 12:19:51.483987 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.484275 26343 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:51.479957 25780 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:19:51.480422 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:51.490958 31099 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.492214 31164 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.505697 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:51.501585 26342 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:51.507165 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:51.510655 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51560.5us
W20260709 12:19:51.512070 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 61252.1us
I20260709 12:19:51.470996 31165 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.494472 31164 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.521528 30811 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:51.520147 31172 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:51.517148 31161 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:51.530707 25751 leader_election.cc:400] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:19:51.531035 25751 leader_election.cc:403] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:19:51.533541 31165 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:51.535372 31165 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:19:51.536325 31024 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.539029 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 2983) took 1073 ms. Trace:
I20260709 12:19:51.539484 30895 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.541262 31103 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:51.542280 30856 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:51.524470 30811 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.542730 30856 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:51.543068 30811 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:51.543889 30856 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.545305 30856 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:51.548714 31152 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.001s
I20260709 12:19:51.549209 31152 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.550407 25785 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:51.550813 26339 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:51.513512 31114 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.532172 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:51.539431 25708 rpcz_store.cc:276] 0709 12:19:50.465836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:50.515804 (+ 49968us) service_pool.cc:224] Handling call
0709 12:19:50.516209 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:50.752638 (+236429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:50.752674 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:50.761290 (+  8616us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:50.761342 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:51.538821 (+777479us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400106154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:51.538862 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:51.538975 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519435520}
W20260709 12:19:51.552445 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 281931us
I20260709 12:19:51.557578 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:51.557783 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:51.556520 31084 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:51.562949 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:51.567104 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:51.567754 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:51.568601 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.570322 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:51.571758 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 234320us
I20260709 12:19:51.572315 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.559399 31084 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.571069 26338 raft_consensus.cc:2376] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 15. Re-sending same reply.
I20260709 12:19:51.566259 31114 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:51.584239 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:51.587262 25638 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:51.571514 31150 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:51.578323 31024 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.589058 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:51.609560 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 103448us
W20260709 12:19:51.610345 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 92422.7us
I20260709 12:19:51.583976 31178 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:51.618222 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.619217 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:51.619439 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 87343us
I20260709 12:19:51.620977 25751 leader_election.cc:400] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:19:51.621351 25751 leader_election.cc:403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:19:51.622725 31177 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:19:51.612624 31161 raft_consensus.cc:476] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 12:19:51.624897 31150 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:51.630532 25753 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.012s	sys 0.006s
W20260709 12:19:51.630928 25753 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.012s	sys 0.006s
I20260709 12:19:51.632733 31161 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:51.618671 31024 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.652806 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 1203) took 1298 ms. Trace:
I20260709 12:19:51.653499 25634 rpcz_store.cc:276] 0709 12:19:50.354082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:50.356647 (+  2565us) service_pool.cc:224] Handling call
0709 12:19:51.652524 (+1295877us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001e91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:51.652709 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2307742080}
I20260709 12:19:51.654573 31035 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:51.655046 30808 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.655187 31035 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.655737 30808 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.657704 31035 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.658355 31179 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.594834 31164 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.654078 25532 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:51.660964 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 63993.8us
W20260709 12:19:51.662642 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 50666us
I20260709 12:19:51.659164 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:51.598119 31165 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.658910 31179 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.665174 31161 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.669767 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.670819 31164 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.672816 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:51.675679 31084 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:51.661513 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44384.5us
I20260709 12:19:51.688627 31084 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:51.690119 31161 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.690593 31164 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.663376 31152 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.698402 31165 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.718734 31178 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.719271 31178 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.720441 31165 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.721442 31178 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.702885 30811 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.677105 31177 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:51.653986 31162 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.686066 25532 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 9, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 9 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:51.664216 30808 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.711298 31025 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.712050 31179 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.712608 31024 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.725560 31152 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.714617 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:51.696674 31015 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:51.736815 25532 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.676290 31093 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.731701 25629 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:51.722297 31084 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:51.737592 25782 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:51.740672 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:51.713205 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2319) took 1900 ms. Trace:
I20260709 12:19:51.680115 30535 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:51.744866 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 59899.8us
I20260709 12:19:51.750382 25785 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:19:51.741458 25636 rpcz_store.cc:276] 0709 12:19:49.813133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:49.813340 (+   207us) service_pool.cc:224] Handling call
0709 12:19:49.813680 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:49.813915 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:49.813940 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:49.829508 (+ 15568us) log.cc:844] Serialized 26 byte log entry
0709 12:19:50.460458 (+630950us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:50.460578 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:50.460720 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:51.108932 (+648212us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:51.691638 (+582706us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b2400121958. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27289c4 00007f29b2726acb 00007f29b28e1dc9 00007f29b28e1a55 00007f29b28e0fb0 00007f29b5557488 00007f29b5557be8 00007f29b552f398 00007f29b552a76a 00007f29b5529bb6
0709 12:19:51.694644 (+  3006us) raft_consensus.cc:1685] finished
0709 12:19:51.694680 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:51.713002 (+ 18322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26325248,"thread_start_us":474901,"threads_started":1,"wal-append.queue_time_us":570297}
I20260709 12:19:51.751770 31162 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:51.751480 25532 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 10, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 10 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:51.760231 25630 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:51.761109 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 88390.2us
I20260709 12:19:51.762229 25630 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:51.752427 31180 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:51.770704 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:51.761659 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75655.5us
I20260709 12:19:51.760771 31162 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:19:51.773800 25532 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.760926 31177 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.768211 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:51.783429 25782 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:51.771960 31150 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
W20260709 12:19:51.770522 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 89153.6us
W20260709 12:19:51.770784 25533 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 299) took 34091 ms (client timeout 15000 ms). Trace:
I20260709 12:19:51.771907 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:51.787034 25603 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:51.790608 25532 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 3 to 8, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 8 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:51.776158 30856 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:51.772677 30895 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:51.796309 30895 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.798316 31025 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.798712 31190 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:51.802729 31185 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.803330 31185 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.773317 31180 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:51.808146 31084 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:51.807912 31180 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.773109 31152 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.811730 31152 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.811823 30535 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.813711 31177 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.795184 31179 raft_consensus.cc:2764] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:51.795526 31114 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.815610 31114 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:51.817893 31114 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:51.819463 30811 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.820333 30856 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:51.820533 25751 leader_election.cc:336] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.727s (SENT)
I20260709 12:19:51.808609 31084 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.821776 31084 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:51.821573 30856 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:51.823011 31059 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:51.823735 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:51.809748 31180 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:51.827191 25603 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:51.826372 25532 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:51.825492 31055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.006s	sys 0.044s
W20260709 12:19:51.842681 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.843861 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:51.838271 25532 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 7 to 11, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 11 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:51.850839 25532 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:51.819490 31093 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.798445 30895 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:51.828588 31185 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:51.848868 31055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.006s	sys 0.044s
I20260709 12:19:51.837630 31059 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:51.868779 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:51.875767 25532 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 4 to 14. New cstate: current_term: 14 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.878127 25532 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 9, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 9 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:51.879976 25532 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 9, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 9 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:51.883121 25751 leader_election.cc:336] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.120s (SENT)
I20260709 12:19:51.883952 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:51.886658 25633 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:51.886425 31055 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:51.890105 25780 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:51.881925 25532 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 9, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:19:51.890863 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 119096us
I20260709 12:19:51.895802 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:51.900058 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:51.902303 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:51.906957 25779 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:51.903606 25778 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:51.909165 31177 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:51.910097 31177 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:51.910894 31177 raft_consensus.cc:2764] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:19:51.921295 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.921972 26342 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:51.922847 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.923478 25786 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:51.924217 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.924875 26344 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:51.925457 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.926379 26339 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:51.929435 25778 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:19:51.935832 25782 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
W20260709 12:19:51.932538 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 142948us
I20260709 12:19:51.938336 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:51.940068 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:51.942921 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.870054 30866 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:51.875828 31093 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:51.955279 25779 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:51.958005 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:51.961396 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:51.962805 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:51.929492 31059 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:51.966214 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:51.969538 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 98801.5us
I20260709 12:19:51.974695 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:51.975520 25631 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:51.976451 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:51.976397 25629 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:51.979698 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:51.963085 31177 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:51.984299 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:51.986810 25635 raft_consensus.cc:2368] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:19:51.971040 31059 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:51.973760 31114 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:51.972682 31025 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:51.998347 25630 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:51.998963 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 175301us
I20260709 12:19:51.999250 25677 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:51.974645 30866 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:52.001784 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.005865 25786 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:51.975903 25750 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:52.002866 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:52.006186 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:52.002434 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:52.012190 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.013238 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.014962 25634 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:52.009343 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:52.021378 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.026666 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.005965 31099 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:52.031042 25638 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:52.032147 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.048619 25637 raft_consensus.cc:2410] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:52.017439 31179 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:52.017062 30856 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:52.055986 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:52.001766 31084 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:52.062050 25786 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:52.063174 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.029783 31177 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:52.047277 31099 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.070688 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.071830 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.076375 25635 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:51.787017 25533 rpcz_store.cc:269] 0709 12:19:17.667870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.670192 (+  2322us) service_pool.cc:224] Handling call
0709 12:19:51.746848 (+34076656us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:41.023861 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:41.024630 (+   769us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:41.024696 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:19:41.024739 (+    43us) tablet.cc:662] Decoding operations
0709 12:19:41.080777 (+ 56038us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:41.107497 (+ 26720us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:41.107540 (+    43us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:19:41.109245 (+  1705us) tablet.cc:701] Row locks acquired
0709 12:19:41.109310 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:19:41.109524 (+   214us) write_op.cc:270] Start()
0709 12:19:41.109694 (+   170us) write_op.cc:276] Timestamp: P: 1783599581109456 usec, L: 0
0709 12:19:41.109738 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:19:41.131720 (+ 21982us) log.cc:844] Serialized 7225 byte log entry
0709 12:19:41.536031 (+404311us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598ae28 00007f29b93c5581 00007f29b93c4d58 00007f29b93c99f1 00007f29b93c99a9 00007f29b93c9939 00007f29b93c9901 00007f29b93c8b2d 0000000000388634 0000000000388479
0709 12:19:42.229379 (+693348us) op_driver.cc:464] REPLICATION: finished
0709 12:19:42.696935 (+467556us) write_op.cc:301] APPLY: starting
0709 12:19:42.708249 (+ 11314us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:43.231494 (+523245us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:43.231576 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:43.774177 (+542601us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:43.774644 (+   467us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:19:43.774730 (+    86us) write_op.cc:312] APPLY: finished
0709 12:19:43.775274 (+   544us) log.cc:844] Serialized 164 byte log entry
0709 12:19:43.775564 (+   290us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:43.775770 (+   206us) write_op.cc:454] Released schema lock
0709 12:19:43.775873 (+   103us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:43.776373 (+   500us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:43.776408 (+    35us) write_op.cc:454] Released schema lock
0709 12:19:43.779060 (+  2652us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b1800396a60. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
Metrics: {"mutex_wait_us":1038520,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":395621,"apply.run_cpu_time_us":37640,"apply.run_wall_time_us":1085082,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79475,"num_ops":18,"prepare.queue_time_us":120473,"prepare.run_cpu_time_us":56412,"prepare.run_wall_time_us":512449,"raft.queue_time_us":281902,"raft.run_cpu_time_us":46290,"raft.run_wall_time_us":420197,"replication_time_us":1034225,"spinlock_wait_cycles":66750464,"thread_start_us":632529,"threads_started":4,"wal-append.queue_time_us":291720,"wal-append.run_cpu_time_us":29401,"wal-append.run_wall_time_us":2600797}]]}
I20260709 12:19:52.081504 25780 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:19:52.081986 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.057364 31179 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:52.072283 31193 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:19:52.126888 25631 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:52.126765 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 121224us
I20260709 12:19:52.027521 31025 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.059887 30856 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:52.065465 31177 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:52.133407 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.140344 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:52.079478 31099 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:52.156719 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.128270 31179 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:52.159739 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:52.161347 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.137502 30856 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:52.147989 31177 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:52.182325 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:52.190601 25786 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:52.186970 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.187788 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.180567 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.192385 25638 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:52.195483 25636 raft_consensus.cc:2410] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:52.197500 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.194183 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:52.193054 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.215368 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.213716 31185 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.216257 25707 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:52.216686 31185 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.154148 31114 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:52.222771 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.224529 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:52.227180 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.229120 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.236943 25712 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:52.238822 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.242074 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2321) took 1824 ms. Trace:
I20260709 12:19:52.245498 31185 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:52.245690 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.246306 31209 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.246995 31209 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.151463 30866 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.248317 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.249459 31209 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:52.249624 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.253788 31035 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:52.254189 31035 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:52.255147 31035 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.256150 31035 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:52.259188 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.259855 25635 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:52.260887 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.261519 25635 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:52.262534 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.263509 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.264802 25708 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:52.265491 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.265842 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.266532 25705 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:52.266891 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:52.267248 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.267766 30808 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:52.236107 31024 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:52.268296 25712 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:19:52.242610 25633 rpcz_store.cc:276] 0709 12:19:50.417663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:50.444673 (+ 27010us) service_pool.cc:224] Handling call
0709 12:19:50.450967 (+  6294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:51.887092 (+1436125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.237890 (+350798us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54001c61d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:52.241996 (+  4106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3300730752}
I20260709 12:19:52.269672 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.269778 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.271311 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.272169 25633 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:52.272921 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.273885 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.268551 31024 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.274824 25706 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:52.275012 31024 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:52.268240 30808 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.278821 30808 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:52.279318 25712 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:52.282212 31201 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:19:52.247045 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:52.286953 31200 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:19:52.287050 25629 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:52.288079 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.288734 25629 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:52.289469 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.290112 25629 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:52.292191 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.273962 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:52.292898 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 55282.6us
I20260709 12:19:52.291455 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:52.250273 25631 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
W20260709 12:19:52.297222 26309 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.605s (SENT)
I20260709 12:19:52.300801 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.294471 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.219599 31150 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:52.303983 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.294631 25638 raft_consensus.cc:2368] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:19:52.304703 25786 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:52.305691 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.310668 25629 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:52.306624 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.310002 25634 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:19:52.311379 25712 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:52.314924 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:52.315759 25786 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:52.317761 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.319730 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.320458 25713 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:52.320971 25636 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:52.326993 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.318681 25635 raft_consensus.cc:2376] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 16. Re-sending same reply.
I20260709 12:19:52.337558 25711 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
W20260709 12:19:52.338682 25751 leader_election.cc:336] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.671s (SENT)
I20260709 12:19:52.339365 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:19:52.341955 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.063s	sys 0.050s
I20260709 12:19:52.339037 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:52.361910 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60265.7us
I20260709 12:19:52.361959 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.367017 25631 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:52.367370 25704 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:19:52.246325 31025 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:52.365392 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:52.308220 31185 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:52.365710 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.365231 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:52.371838 25638 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:52.374080 31059 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.375130 25633 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:19:52.228122 31198 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.138s
W20260709 12:19:52.364372 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 523482us
W20260709 12:19:52.378630 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.401087 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.404281 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.405889 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.381963 31152 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:52.415782 25712 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:52.411695 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.382179 31185 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.398636 31210 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.433748 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.456348 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:52.443142 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.446820 31114 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.415135 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:52.460328 25782 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:52.461585 25629 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:52.452893 25706 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:52.458245 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.458436 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.463193 25707 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:52.463945 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.362113 31208 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:19:52.464689 25712 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:19:52.466123 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58575.8us
I20260709 12:19:52.463593 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:52.408931 31084 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:52.483930 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:52.454285 31190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.026s	sys 0.023s
I20260709 12:19:52.504993 25778 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:52.457789 31139 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:19:52.517956 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.500816 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:52.417140 31158 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.455457 31185 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:52.525061 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.526872 25711 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:52.485209 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:52.529424 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.489334 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.468380 31210 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:52.532519 25704 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:52.459365 31162 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:19:52.468428 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.551654 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:52.554831 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 189543us
I20260709 12:19:52.530407 25712 raft_consensus.cc:2376] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16. Re-sending same reply.
I20260709 12:19:52.544305 25782 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:52.521391 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:52.514251 25601 connection.cc:256] RPC call timeout handler was delayed by -1.1806391150000763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2759s for a response
W20260709 12:19:52.566742 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 99483.4us
I20260709 12:19:52.517925 31208 consensus_queue.cc:1243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:19:52.505223 31190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.026s	sys 0.023s
W20260709 12:19:52.583112 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 285915us
W20260709 12:19:52.586089 26309 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.614s (SENT)
I20260709 12:19:52.586529 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.484867 31024 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.584009 25713 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:52.588784 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.583818 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.587990 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:52.591277 25705 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:52.591691 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.594059 25711 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:19:52.597470 25712 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:19:52.599185 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.603434 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:52.602777 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 193731us
I20260709 12:19:52.594403 25779 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:52.600709 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.605271 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.607347 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.608129 25713 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:52.609154 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 270222us
W20260709 12:19:52.609923 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.612227 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.612592 25712 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:52.528420 31158 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:52.588197 31190 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:52.635526 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:52.592599 31216 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:52.597972 31207 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:52.596412 31208 raft_consensus.cc:2764] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:52.645594 25710 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:52.652632 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 196370us
I20260709 12:19:52.653690 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.655810 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:52.656860 25704 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:52.657003 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:52.658772 31219 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:52.659214 31219 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.659525 31219 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:52.660252 25713 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:52.662374 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:52.655218 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 154547us
W20260709 12:19:52.665701 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52970.7us
I20260709 12:19:52.596297 30856 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.671752 25704 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:19:52.672113 25706 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:52.673264 25629 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:52.683270 25710 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:52.654381 31150 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:52.688349 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:52.650871 31207 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:52.719380 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:52.721403 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46396.4us
W20260709 12:19:52.731247 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 43561us
I20260709 12:19:52.638075 31099 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:52.763048 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.688279 31059 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:52.719517 31015 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:52.720180 31208 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.654675 31077 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.237s
I20260709 12:19:52.759907 31198 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:52.783605 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:52.795831 25711 raft_consensus.cc:2368] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15.
I20260709 12:19:52.783680 25709 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:52.722908 31207 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:52.788019 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 266442us
I20260709 12:19:52.789088 31216 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:19:52.789456 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.804143 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.805495 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.725701 30808 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.810243 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:52.817163 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 246611us
I20260709 12:19:52.821455 25705 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:19:52.821952 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.790962 31225 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:52.803169 31198 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:52.826661 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 240605us
I20260709 12:19:52.826176 31024 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:52.830152 31226 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:52.802739 30870 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.807746 31162 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:52.803943 31216 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:52.835265 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54024.8us
I20260709 12:19:52.838287 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.811657 31015 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:52.829536 26307 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:52.806038 31208 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:52.837224 31198 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:52.835210 31103 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:52.828334 31225 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.857079 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:52.905179 25629 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:52.908963 25707 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
W20260709 12:19:52.858647 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48523.4us
W20260709 12:19:52.868604 31214 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:19:52.845367 31162 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:52.854691 30870 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:52.875988 31223 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 12:19:52.879487 30808 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:52.877381 31222 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.827711 31190 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.879382 30856 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:52.879875 31210 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:52.880409 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 184669us
I20260709 12:19:52.823587 31059 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:19:52.881234 31158 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:52.867206 31224 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:52.880817 31216 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.870163 31015 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:52.886777 25779 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
W20260709 12:19:52.900357 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 81955.6us
I20260709 12:19:52.911455 25712 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:52.910930 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:52.911903 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:52.914455 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:52.937140 25636 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:52.916642 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 77234.1us
W20260709 12:19:52.876624 31211 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:19:52.840330 31220 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:19:52.885025 31103 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:52.917649 31180 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 1.079s	user 0.015s	sys 0.000s
W20260709 12:19:52.923843 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 118399us
W20260709 12:19:52.913398 31179 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.005s
I20260709 12:19:52.908224 31225 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:52.919251 31177 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.000s
I20260709 12:19:52.927369 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:52.928925 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 79205.7us
I20260709 12:19:52.923763 31024 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:52.952126 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.946475 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:52.959684 25637 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:52.960222 31180 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:52.961457 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.964020 31180 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:52.964505 31180 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:52.964581 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.965490 31084 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:52.965828 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:52.965435 31180 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:52.968838 31213 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:52.936455 31150 raft_consensus.cc:2764] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:19:52.960529 31227 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.971729 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:52.972263 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:52.938700 31236 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.942725 31099 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:52.991694 31218 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:52.960906 31158 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:52.994184 25676 leader_election.cc:400] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9.
I20260709 12:19:52.994571 25676 leader_election.cc:403] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:52.995743 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:52.996848 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:19:52.997160 31035 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.000s
I20260709 12:19:52.952839 31220 consensus_queue.cc:1243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:52.973412 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.006094 26337 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:52.974797 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.007408 26339 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:52.956843 31179 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:52.978241 26344 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:52.978592 31229 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:53.015946 31239 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:53.016462 30808 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:53.017236 30808 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:52.979157 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:52.979595 31232 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:52.980011 31231 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:53.018862 31208 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:52.993436 31196 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:19:53.020479 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:52.973986 31024 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:53.021206 31209 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9.
I20260709 12:19:52.962450 31177 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.023077 31232 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.025465 31177 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:52.990399 31245 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:53.045975 25636 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:52.997923 31227 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.001940 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:53.054670 26336 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:53.005550 31218 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:53.015439 31238 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:52.970256 31224 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:53.015671 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53598.1us
I20260709 12:19:53.025650 31232 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:53.029268 26342 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:53.064484 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:53.030752 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:53.033658 26343 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:52.972960 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.029954 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:52.937988 31222 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:53.070418 31212 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.588s	user 0.003s	sys 0.002s
I20260709 12:19:52.997313 31237 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:19:52.993503 30870 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.071566 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:52.998193 31216 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:53.047340 31208 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:53.078022 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:53.087569 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 141207us
I20260709 12:19:53.089087 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.080392 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:53.028481 31213 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:53.095615 31213 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.073215 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.053856 30856 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:53.091874 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:53.099678 30856 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:53.100024 25629 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:53.100397 30856 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:53.101301 31226 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:53.102017 31204 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:53.091962 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:53.107153 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 132223us
W20260709 12:19:53.109046 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 137951us
I20260709 12:19:52.961346 31162 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.022118 31024 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.082319 31180 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:53.089347 31185 consensus_meta.cc:287] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.000s
W20260709 12:19:53.114194 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 84321.7us
I20260709 12:19:53.124349 26342 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:53.106099 31226 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:53.005175 31236 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.102190 31234 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:19:53.082165 31059 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:19:53.129274 31226 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:53.109774 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:53.139272 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:53.130519 31219 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.143180 31196 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:53.144727 31219 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.148255 26336 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:53.152694 25629 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:53.153889 25677 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:53.119556 31180 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:53.186476 31257 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:53.187067 31257 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:53.187943 31257 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.189255 31257 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:53.199707 26336 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:53.201208 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.201532 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.201874 26341 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:53.202970 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:53.203810 31244 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:19:53.205670 31235 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.158s	user 0.000s	sys 0.002s
I20260709 12:19:53.206599 31198 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.209113 31198 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.210626 31233 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:53.182981 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46075.6us
I20260709 12:19:53.228829 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:53.221909 31059 consensus_queue.cc:1243] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:19:53.224320 30866 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:53.230233 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.129861 31185 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.211721 31204 raft_consensus.cc:2804] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:53.184597 31035 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.211936 31180 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:53.239849 26338 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:53.244246 25631 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:53.245741 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:53.246740 26341 raft_consensus.cc:2368] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12.
I20260709 12:19:53.251152 26336 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:53.251606 26336 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:53.252449 26336 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.253137 31208 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:53.253794 26336 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:53.254406 31208 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:53.254931 31208 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:53.256000 31208 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.256513 30856 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.257647 31208 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:53.258693 31077 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:53.259184 31077 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:53.259277 31179 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:53.260988 30856 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:53.231344 31162 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:53.232379 31233 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.265372 31227 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.266618 31077 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:53.270916 26336 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:53.275352 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:53.276180 26336 raft_consensus.cc:2368] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:53.267508 31222 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:53.119109 30808 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.280709 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:53.282331 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.288625 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.288820 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:53.289390 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.289769 25713 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:53.295789 26342 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 11. Ops: []
I20260709 12:19:53.299021 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.302933 26341 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:53.305190 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.305956 25707 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:53.302996 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.307190 25705 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:53.308336 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.309129 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.269363 31233 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:53.278591 31204 raft_consensus.cc:697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:53.314244 31258 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:53.279296 31035 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.201678 31218 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.300719 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:53.326969 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.342958 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:53.343678 26341 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
W20260709 12:19:53.346225 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45609.5us
I20260709 12:19:53.350095 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:53.350667 25631 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:53.356431 31035 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.335330 30866 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.282183 31207 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.364209 31207 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.361788 30866 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.375064 30866 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.376684 31077 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.377559 31226 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.341751 31114 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.385751 25631 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:53.390838 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:53.387290 31114 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.395906 31114 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.396126 25603 leader_election.cc:400] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15.
I20260709 12:19:53.396456 25603 leader_election.cc:403] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:53.397233 25704 raft_consensus.cc:2410] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:19:53.398955 31185 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:53.399199 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:53.399767 31236 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:53.400007 25637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2564) took 1346 ms (client timeout 1504 ms). Trace:
I20260709 12:19:53.400442 31027 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:53.401202 31204 consensus_queue.cc:237] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.391592 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:53.372514 31254 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:53.407859 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:53.408439 25637 rpcz_store.cc:269] 0709 12:19:52.053382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.060429 (+  7047us) service_pool.cc:224] Handling call
0709 12:19:53.399826 (+1339397us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54002a5b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:53.399942 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037842816}
I20260709 12:19:53.413856 26335 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:53.423363 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.424796 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:53.435701 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:53.436587 25637 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:53.443435 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:53.451625 25637 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:53.366562 31225 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.399212 31226 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.461325 31035 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:53.461951 31035 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.462733 26340 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:53.463831 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:53.464159 31035 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.464686 25638 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:53.471590 31077 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:53.415719 31268 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15.
I20260709 12:19:53.456277 31185 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:53.491351 31185 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:53.492390 31185 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.502664 31254 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:53.503998 31254 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.463521 31180 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.455653 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:53.522944 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 67422.3us
I20260709 12:19:53.525242 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2719) took 1217 ms. Trace:
I20260709 12:19:53.525774 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:53.528381 25710 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:19:53.528633 25712 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:53.531221 25638 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:53.532873 25709 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:53.543915 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.544358 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:53.544621 25629 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:53.544114 25705 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:19:53.550027 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:53.570401 26339 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:53.573848 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.580768 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.530849 31227 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:53.521883 31239 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:53.577564 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:53.578718 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:53.521682 31258 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:53.547552 31152 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.581851 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.585705 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:53.547387 26344 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:53.595782 25676 leader_election.cc:400] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12.
I20260709 12:19:53.595950 30870 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:53.597052 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:53.598771 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:53.602259 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.602715 31250 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:53.603752 31250 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:53.604231 31250 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:53.604720 31250 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:53.613420 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:53.571699 31226 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:53.619801 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.614344 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.543255 31234 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.580324 31027 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:53.592955 30856 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:53.580030 31213 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:19:53.576906 31114 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:19:53.633832 31180 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.635840 31225 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:53.635972 31271 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:19:53.636838 31267 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:19:53.636435 31226 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.615800 31227 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:53.644788 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.646214 31275 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:53.647207 31258 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.655084 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:53.671444 25705 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:53.671732 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 73048.5us
I20260709 12:19:53.611865 31152 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.651885 30856 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:53.708433 31237 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:53.754927 30856 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:53.709587 25676 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:53.711722 31226 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.525686 25780 rpcz_store.cc:276] 0709 12:19:52.303335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.312474 (+  9139us) service_pool.cc:224] Handling call
0709 12:19:52.364488 (+ 52014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:52.364702 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:52.548822 (+184120us) log.cc:844] Serialized 14 byte log entry
0709 12:19:52.976344 (+427522us) log.cc:844] Serialized 14 byte log entry
0709 12:19:53.519797 (+543453us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b1800323860. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:19:53.519983 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:53.520053 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:53.520120 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.520224 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.520393 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88918,"prepare.queue_time_us":277008,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":79,"raft.queue_time_us":622645,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":35016,"spinlock_wait_cycles":17293568,"thread_start_us":490623,"threads_started":2,"wal-append.queue_time_us":438727}
I20260709 12:19:53.758986 31229 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:53.759617 31216 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.671s	user 0.008s	sys 0.004s
W20260709 12:19:53.716221 31246 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.647s	user 0.003s	sys 0.002s
I20260709 12:19:53.762454 31077 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:53.763646 31077 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:53.763281 31103 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.764052 31077 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:53.764605 31103 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:53.719408 31180 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:53.736853 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:53.694954 31208 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:53.709154 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64495.4us
I20260709 12:19:53.766038 31180 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:53.658699 31055 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:53.766573 31180 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:53.766889 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:53.767623 31180 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.763391 31275 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:19:53.770381 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 92204.4us
I20260709 12:19:53.771705 31226 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:19:53.771942 31269 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:19:53.772114 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:53.773303 31084 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:53.774021 31247 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:53.774811 31247 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:53.775503 31204 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:53.775768 31247 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:53.775951 31204 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:53.776175 31247 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:53.653461 31234 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.776973 31247 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.764948 31077 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.759554 31229 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:53.772258 31226 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:53.717628 31219 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:19:53.715085 30870 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:53.770203 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.803370 25750 leader_election.cc:400] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:53.794469 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:53.794946 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:53.837198 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 63759.8us
I20260709 12:19:53.837929 25750 leader_election.cc:403] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:19:53.828527 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.839177 25635 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:19:53.795981 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 686241us
W20260709 12:19:53.839360 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.842075 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:53.786026 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 696955us
I20260709 12:19:53.846117 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.848503 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.779409 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:53.850514 25637 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:53.850425 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:53.780957 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.855815 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.780052 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:53.794776 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:53.857558 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 77595us
I20260709 12:19:53.857870 26337 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:53.784541 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:53.799767 25632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2547) took 1841 ms (client timeout 2120 ms). Trace:
I20260709 12:19:53.859105 25638 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:53.863683 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 93551us
I20260709 12:19:53.865058 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:53.865612 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:53.859328 25632 rpcz_store.cc:269] 0709 12:19:51.958744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.966903 (+  8159us) service_pool.cc:224] Handling call
0709 12:19:53.799521 (+1832618us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54001c5cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:53.799697 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3682037632}
W20260709 12:19:53.864749 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 61632.9us
I20260709 12:19:53.857973 25630 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:53.877398 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.804510 31251 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:53.879596 25632 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 11. Ops: []
I20260709 12:19:53.880224 25636 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:19:53.887104 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:53.887281 25677 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:53.880796 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:53.887936 25784 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:53.882146 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.888908 25711 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:53.883124 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.884117 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:53.790270 31158 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.786545 31238 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:53.778431 31247 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:53.892690 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:53.909216 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:53.909708 25780 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:53.803795 31231 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.804021 31162 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.804247 31177 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.855556 31226 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:53.900264 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:53.888335 31268 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:53.902691 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52344.4us
I20260709 12:19:53.949214 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.903930 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:53.904146 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:53.804800 31249 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:19:53.785336 31224 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:53.782786 31084 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:53.790510 31225 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:53.778474 31234 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:53.904248 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:53.872931 31257 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:53.804922 31239 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 12:19:53.790069 31179 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.894500 31219 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:53.759627 31227 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:53.939182 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.948174 25711 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:19:53.950736 25778 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
W20260709 12:19:53.905586 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.898160 31035 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:53.872710 31208 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:19:53.898402 31245 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.891690 31252 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:53.958469 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:53.906577 31236 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.931070 31114 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:53.949424 31280 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:19:53.776916 31204 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.895117 31216 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:53.971946 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:53.991565 25780 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:53.973475 31238 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:53.933137 31231 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.908922 31158 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:53.980288 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78248.4us
W20260709 12:19:54.009455 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 133218us
I20260709 12:19:53.995311 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:54.061733 25778 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:53.997509 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:54.062731 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 65300.7us
I20260709 12:19:54.063486 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:53.972525 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 85304.7us
I20260709 12:19:54.064379 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:53.972163 31219 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:53.971520 31257 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:53.948879 31177 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.953904 31162 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.077991 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:54.078835 26307 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:53.964501 31225 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.956202 31268 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:53.989198 31114 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:53.992946 31208 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:53.964816 31233 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:19:53.988333 31234 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:53.971702 31179 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:54.096203 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 147059us
I20260709 12:19:54.075963 31219 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:54.075734 31218 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:54.083405 31035 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:54.094937 31204 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:19:54.067749 31196 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:19:54.092167 31208 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:54.148604 25783 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:54.092478 31114 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:54.059703 31236 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:53.985365 31245 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.123114 31240 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.165505 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:19:54.201290 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 123373us
W20260709 12:19:54.152046 31250 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.003s
I20260709 12:19:54.204735 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:54.220016 25784 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:54.151378 31291 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:54.153905 31251 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:54.152678 31290 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:54.217715 31027 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.000s
I20260709 12:19:54.160986 31233 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:54.229254 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 164858us
I20260709 12:19:54.209302 31238 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.236963 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.223825 25780 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
W20260709 12:19:54.243903 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78459.2us
I20260709 12:19:54.245328 25751 leader_election.cc:400] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:19:54.127846 31224 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:54.247946 31287 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:19:54.255344 31225 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.256059 31225 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.221205 31035 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:54.221844 31292 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:54.134315 31084 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.232234 31152 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:54.231424 31290 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.240697 31027 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:54.240100 31233 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:54.256269 31249 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:54.257234 31218 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:54.282905 31035 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:54.283428 31035 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.283195 31218 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.283895 31035 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:54.257541 31234 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:54.282008 31290 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:54.222673 31250 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.267066 31292 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:54.294965 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:54.302001 26343 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:54.256645 31225 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.298830 25629 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:54.299649 26340 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:54.301023 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52903.8us
W20260709 12:19:54.293916 31282 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:19:54.230685 31257 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.287818 31233 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:54.309588 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:54.226204 31240 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.322863 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:54.328787 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.329578 25712 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:19:54.330605 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.331279 25712 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 18: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:54.293227 31234 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:19:54.335528 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.303539 31249 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:54.276141 31238 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.341934 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.342871 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.346823 25708 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:19:54.334053 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:54.270310 31208 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.291787 31291 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:19:54.338981 31231 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.352164 31158 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:54.312186 31302 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:19:54.355664 31236 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:54.360486 31222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.005s	sys 0.022s
I20260709 12:19:54.381289 31294 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:54.406512 31294 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.406909 31294 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:54.407997 31240 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.408738 31245 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:54.382176 31296 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 12:19:54.382721 31284 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 12:19:54.359328 31268 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:54.383656 31285 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 12:19:54.384276 31283 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 12:19:54.367390 31224 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:54.414556 31224 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:54.415706 31224 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.416429 31294 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:54.387861 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78402.8us
W20260709 12:19:54.418771 25601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.014s	sys 0.019s
W20260709 12:19:54.419159 25601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.014s	sys 0.019s
W20260709 12:19:54.386441 30856 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.631s	user 0.005s	sys 0.005s
I20260709 12:19:54.402645 31225 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:54.403348 31289 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:19:54.418532 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:54.395915 31288 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
W20260709 12:19:54.420100 31222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.005s	sys 0.023s
I20260709 12:19:54.323267 31251 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.428037 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:54.428808 25784 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:54.431236 31225 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:54.431713 31225 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:54.432109 31225 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:54.432184 31238 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:54.435426 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2734) took 1033 ms. Trace:
I20260709 12:19:54.435798 25779 rpcz_store.cc:276] 0709 12:19:53.402283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.405788 (+  3505us) service_pool.cc:224] Handling call
0709 12:19:53.414966 (+  9178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:53.787523 (+372557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.787559 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:53.788186 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.109984 (+321798us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:54.122674 (+ 12690us) log.cc:844] Serialized 14 byte log entry
0709 12:19:54.122898 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.122977 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:54.435207 (+312230us) raft_consensus.cc:1685] finished
0709 12:19:54.435231 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.435363 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":372134,"prepare.queue_time_us":12449,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":506030,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":187,"thread_start_us":301907,"threads_started":1,"wal-append.queue_time_us":443596}
I20260709 12:19:54.437862 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:54.438832 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.439225 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.442651 31234 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.429682 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:54.446827 25779 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:54.447258 31309 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:54.447956 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:54.447810 31309 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.457626 25778 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
W20260709 12:19:54.451787 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.450924 25705 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:54.451942 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:54.478183 25708 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:19:54.451025 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:54.402480 31103 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.003s
I20260709 12:19:54.455168 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.482996 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.491370 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:54.499119 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 422195us
I20260709 12:19:54.499233 25710 raft_consensus.cc:2410] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:54.500356 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:54.450168 31309 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:54.451120 31302 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.450098 31226 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.488358 31294 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:54.518394 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67442.3us
I20260709 12:19:54.518385 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.518774 25778 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:54.518599 25706 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:19:54.549166 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.553012 25706 pending_rounds.cc:85] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 8
I20260709 12:19:54.553272 25706 pending_rounds.cc:107] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:19:54.553473 26340 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:54.553594 25706 raft_consensus.cc:2889] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:54.504267 31291 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:54.553995 31077 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:54.554942 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3010) took 1944 ms. Trace:
I20260709 12:19:54.554584 31291 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.555341 25706 rpcz_store.cc:276] 0709 12:19:52.610156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.820789 (+210633us) service_pool.cc:224] Handling call
0709 12:19:52.841962 (+ 21173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:54.554090 (+1712128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.554680 (+   590us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b5400073fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:54.554855 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3824860416}
I20260709 12:19:54.557155 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:54.557581 31291 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.453542 31233 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.559029 26337 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:54.559973 31233 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:54.454640 30856 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.502166 31196 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:54.561100 25707 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:54.568249 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:54.570284 26307 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:54.571612 31103 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.547518 31231 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:54.584652 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.425913 31249 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.552770 31238 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:54.591967 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.590525 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:54.572271 31196 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:54.587566 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:54.581529 31252 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.003s
I20260709 12:19:54.581781 31233 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:54.616057 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.587095 31231 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.586655 31250 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.517494 31302 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.624290 31231 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:54.624012 31233 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:54.625872 31233 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:54.626711 31302 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.627974 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:54.628813 25629 raft_consensus.cc:2368] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:54.629958 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.598335 31238 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.632591 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.633790 26307 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:54.594589 31236 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.641865 25783 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:19:54.641990 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.642469 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.644399 31035 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.646772 31035 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:54.647189 31270 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.588964 31103 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.647738 31270 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.631667 31245 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:54.648842 31177 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.649161 31245 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:54.649593 31245 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:54.650048 26340 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:54.646809 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:54.650738 31320 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:54.649731 31103 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.662102 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.640268 31218 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:54.596704 30856 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.670650 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.671192 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:54.666059 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:54.672328 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.664899 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.673379 25707 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:54.679870 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:54.680250 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.680368 25637 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:54.621506 31252 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:54.683060 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.668330 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:54.683380 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.628841 31226 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:54.717216 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:54.685948 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:54.682197 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.715106 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.721889 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.722545 25705 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:54.725083 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:54.715382 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.649718 31270 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.728044 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.728835 25636 raft_consensus.cc:2393] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:54.716111 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:54.682600 31314 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:19:54.729764 26338 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:54.733819 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:54.723697 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:54.726332 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:54.734798 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.736606 25710 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:54.679718 31103 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:54.685115 31250 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:54.742841 26339 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:54.747802 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:54.755209 25710 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:54.666987 31236 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.724234 31204 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:19:54.744565 31313 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.752463 30870 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:54.773490 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.776196 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:54.778023 26337 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:54.750697 31315 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:54.751282 31103 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.738552 30856 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:19:54.759243 31281 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.759109 31213 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.762316 31185 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:54.744939 31196 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.755059 31226 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:19:54.756304 31290 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.761358 31275 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:19:54.755118 31250 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.790036 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:54.808928 26344 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:54.766100 31258 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:19:54.768059 31303 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:19:54.825263 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:54.808969 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.576s (SENT) [suppressed 25 similar messages]
I20260709 12:19:54.835098 25750 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:54.780925 31318 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:54.788681 31320 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:54.823228 31311 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
I20260709 12:19:54.793439 31162 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:54.789718 31315 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:54.851677 25705 pending_rounds.cc:85] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 9
I20260709 12:19:54.861822 25705 pending_rounds.cc:107] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:19:54.863525 25705 raft_consensus.cc:2889] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:54.848702 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:54.840346 31177 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:54.828364 31291 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:54.832331 30856 consensus_queue.cc:1243] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 10d0c7349085463595aedaf7c071e742 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:54.855340 31252 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:54.847338 31318 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:54.922154 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 73591.3us
I20260709 12:19:54.805100 31213 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.778885 31313 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:54.879253 31321 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 12:19:54.882077 31316 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:19:54.929862 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:54.887374 31322 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:19:54.934696 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3032) took 1014 ms. Trace:
I20260709 12:19:54.935117 25711 rpcz_store.cc:276] 0709 12:19:53.920518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.990090 (+ 69572us) service_pool.cc:224] Handling call
0709 12:19:54.069364 (+ 79274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.164914 (+ 95550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.164963 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.216882 (+ 51919us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.333079 (+116197us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:54.333147 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.333246 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:54.934431 (+601185us) raft_consensus.cc:1685] finished
0709 12:19:54.934456 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.934629 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9238,"raft.queue_time_us":158099,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6553,"thread_start_us":161812,"threads_started":2,"wal-append.queue_time_us":149118}
I20260709 12:19:54.792662 31281 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:54.886677 31114 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.004s
I20260709 12:19:54.939191 31114 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.940176 25712 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:54.941455 31114 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:54.970459 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 998) took 1085 ms. Trace:
I20260709 12:19:54.888216 31185 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.985373 25708 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
W20260709 12:19:54.996928 31225 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 12:19:54.997442 31225 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:54.998674 25778 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:54.998828 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:54.999491 25708 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:55.000792 31103 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:54.935774 30870 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:55.009101 26335 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:55.010075 25787 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:54.992072 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:55.005878 30870 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.973570 31290 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:54.951495 31299 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.256s
I20260709 12:19:54.968247 31250 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:54.952539 31326 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:19:55.014848 31239 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:55.015296 31239 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:55.016271 31239 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.017498 31239 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:55.019987 31251 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:55.020524 31251 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:55.021888 31219 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.890s	user 0.006s	sys 0.006s
I20260709 12:19:55.021445 31251 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.986501 25785 rpcz_store.cc:276] 0709 12:19:53.885504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.322277 (+436773us) service_pool.cc:224] Handling call
0709 12:19:54.970236 (+647959us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54001a72d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:54.970390 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000455040}
W20260709 12:19:54.955752 31319 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
W20260709 12:19:54.953449 31324 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:19:54.931150 31162 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:54.928571 31318 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:54.857203 31320 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:54.931490 31177 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 12:19:54.977077 31249 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:55.082504 31249 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:55.084183 31247 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 1.155s	user 0.012s	sys 0.000s
W20260709 12:19:55.085147 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 276191us
I20260709 12:19:55.085570 25709 raft_consensus.cc:2410] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:19:55.071203 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.060187 31238 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:54.987551 31303 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.088933 25633 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:55.025170 31213 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.057894 31281 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.027096 31313 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.031801 31236 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.100997 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.102190 25785 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:19:55.122251 25676 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.626s (SENT)
I20260709 12:19:55.074666 31218 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:55.022445 31219 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.034701 31185 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.133311 26339 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:55.134636 31219 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.134843 31185 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.128310 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.089736 31250 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.135838 31250 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.145177 31084 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:55.145673 31084 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:55.150669 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 990) took 1268 ms. Trace:
I20260709 12:19:55.151098 25779 rpcz_store.cc:276] 0709 12:19:53.882756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.448871 (+566115us) service_pool.cc:224] Handling call
0709 12:19:54.459749 (+ 10878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.463255 (+  3506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:54.463302 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.511782 (+ 48480us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.679110 (+167328us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:54.738820 (+ 59710us) log.cc:844] Serialized 14 byte log entry
0709 12:19:54.739062 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.739135 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.150394 (+411259us) raft_consensus.cc:1685] finished
0709 12:19:55.150417 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.150594 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17663,"prepare.queue_time_us":59954,"raft.queue_time_us":3183,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":25957,"spinlock_wait_cycles":7734144,"thread_start_us":99133,"threads_started":1,"wal-append.queue_time_us":237906}
I20260709 12:19:55.152587 31320 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.088670 31231 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.131562 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:55.128937 31270 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.126667 31247 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:55.167457 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.171644 25711 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:19:55.170845 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 227817us
I20260709 12:19:55.175410 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.176855 26307 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:55.167783 31233 consensus_meta.cc:287] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
I20260709 12:19:55.172497 31290 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:55.022836 31315 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.181223 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.169323 31249 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.191726 25711 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:19:55.193157 25711 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:55.194334 26341 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:55.193413 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:55.196647 31177 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:55.165731 31084 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.198117 31084 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:55.214044 31245 consensus_meta.cc:287] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:19:55.179250 31247 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.218055 31247 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.172719 31270 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.219060 31162 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:55.220496 31270 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.197206 31177 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.183969 31218 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:55.093833 31303 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:55.232321 31329 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:55.186200 31233 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:55.236853 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 138471us
I20260709 12:19:55.227515 31177 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:55.188570 31300 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.227285 31252 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:55.224155 31245 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:55.238592 31317 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.498s	user 0.004s	sys 0.000s
W20260709 12:19:55.250308 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.243577 31290 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.252452 31224 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:55.268903 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:55.270879 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.269477 31337 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.249055 31247 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:55.273252 31290 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.256997 31300 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.277120 31337 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.284261 31258 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.309937 31258 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.311239 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 987) took 1429 ms. Trace:
I20260709 12:19:55.312024 31258 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.315294 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 986) took 1433 ms. Trace:
I20260709 12:19:55.315582 31337 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.315687 25783 rpcz_store.cc:276] 0709 12:19:53.881599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.884242 (+  2643us) service_pool.cc:224] Handling call
0709 12:19:55.315124 (+1430882us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400257bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:55.315231 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549000576}
I20260709 12:19:55.316915 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.317101 31337 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.318395 31084 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:55.318945 31084 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.320349 31234 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:55.321099 31084 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.311640 25781 rpcz_store.cc:276] 0709 12:19:53.881899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.882610 (+   711us) service_pool.cc:224] Handling call
0709 12:19:53.904822 (+ 22212us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:54.358431 (+453609us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:54.374295 (+ 15864us) log.cc:844] Serialized 14 byte log entry
0709 12:19:54.520885 (+146590us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:54.571473 (+ 50588us) log.cc:844] Serialized 52 byte log entry
0709 12:19:54.571634 (+   161us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:54.571670 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.237985 (+666315us) spinlock_profiling.cc:243] Waited 9.26 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:55.310897 (+ 72912us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b5400320854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:55.310952 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.311008 (+    56us) raft_consensus.cc:1685] finished
0709 12:19:55.311021 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.311176 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80313,"prepare.queue_time_us":15679,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":51053,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":14300,"spinlock_wait_cycles":1042203648,"thread_start_us":139448,"threads_started":1,"wal-append.queue_time_us":219241}
I20260709 12:19:55.324268 31231 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.324999 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:55.325455 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 988) took 1443 ms (client timeout 1564 ms). Trace:
W20260709 12:19:55.325847 25787 rpcz_store.cc:269] 0709 12:19:53.882173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.884509 (+  2336us) service_pool.cc:224] Handling call
0709 12:19:55.325297 (+1440788us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b5400258ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:55.325400 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780344704}
I20260709 12:19:55.326917 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.327105 31318 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.328480 26344 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:55.286316 25635 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:55.331112 25710 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:55.331542 25710 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:55.332675 25710 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.333679 25710 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:55.288672 25751 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50966 dropped due to backpressure: service queue is full with 50 items [suppressed 3 similar messages]
I20260709 12:19:55.327505 25787 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:55.303598 25708 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:55.289709 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.318252 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.303776 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.340288 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:55.346789 25786 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:55.336954 25779 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:55.340951 25712 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:19:55.348080 25712 pending_rounds.cc:85] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 10
I20260709 12:19:55.348317 25712 pending_rounds.cc:107] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:19:55.348582 25712 raft_consensus.cc:2889] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:55.343101 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.345291 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.325639 25781 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:55.361027 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:55.361465 25633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 2365) took 1568 ms (client timeout 1899 ms). Trace:
I20260709 12:19:55.361894 25787 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:55.363193 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.363505 31320 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.363992 25787 raft_consensus.cc:2376] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13. Re-sending same reply.
I20260709 12:19:55.365373 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.279791 31290 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.305244 31335 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.367300 31335 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.368371 31218 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:55.369436 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.370229 25781 raft_consensus.cc:2368] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:19:55.366101 25787 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:55.371134 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.371299 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.371883 25781 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:55.296694 31303 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.372761 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.364113 31320 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.373546 25787 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:55.374408 26342 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:55.317600 31337 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:55.375357 25633 rpcz_store.cc:269] 0709 12:19:53.793045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.794584 (+  1539us) service_pool.cc:224] Handling call
0709 12:19:55.361293 (+1566709us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001d6ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:55.361400 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2757668608}
I20260709 12:19:55.385496 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.372870 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.386366 25712 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:55.386874 25781 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:55.392727 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:55.394253 25704 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:55.519668 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:55.291378 31233 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.534020 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2209) took 1030 ms. Trace:
I20260709 12:19:55.535750 25710 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:55.539300 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.540282 25787 raft_consensus.cc:2410] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:19:55.534502 25778 rpcz_store.cc:276] 0709 12:19:54.508059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.556862 (+ 48803us) service_pool.cc:224] Handling call
0709 12:19:55.533007 (+976145us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54000de3d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:55.533282 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013539072}
I20260709 12:19:55.290242 31245 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.364168 31318 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.304881 31300 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.552747 25635 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:55.370397 31335 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.287756 31229 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.560986 31346 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:55.561633 25782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2733) took 2149 ms (client timeout 1988 ms). Trace:
I20260709 12:19:55.383816 31224 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:55.572443 25786 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:55.335146 25751 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50966 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2235)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 1005)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2229)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2241)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2223)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2227)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2742)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2743)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2746)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2747)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2749)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2214)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2750)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2216)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2217)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2222)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2755)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2757)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2236)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2237)
I20260709 12:19:55.580739 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:55.304952 31247 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:55.556135 31301 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 12:19:55.584980 25782 rpcz_store.cc:269] 0709 12:19:53.401963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.409394 (+  7431us) service_pool.cc:224] Handling call
0709 12:19:55.544368 (+2134974us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b54000b04d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:55.550816 (+  6448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2040560896}
I20260709 12:19:55.630523 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:55.631342 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:55.634209 25633 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:55.655040 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.656498 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 1001) took 1021 ms. Trace:
I20260709 12:19:55.658188 25783 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
W20260709 12:19:55.659093 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.659907 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.660450 25704 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:55.640275 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:55.519552 31219 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:19:55.530021 31303 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:19:55.661644 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.664465 25704 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:55.657012 25786 rpcz_store.cc:276] 0709 12:19:54.634870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.303486 (+668616us) service_pool.cc:224] Handling call
0709 12:19:55.656372 (+352886us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.671093 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.671967 25786 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:55.573197 31299 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:55.673787 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.687129 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.674835 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:55.687924 25704 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:55.560806 31177 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.691767 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.693538 25782 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:55.689666 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.631960 31204 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.695353 25786 raft_consensus.cc:3060] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:55.641935 31103 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:55.708364 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 317221us
I20260709 12:19:55.709252 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:55.712546 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.713213 25782 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:55.674906 31258 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.713443 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.724078 25783 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
W20260709 12:19:55.725274 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.676788 31335 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:55.712597 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:55.730069 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:55.732152 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.732030 26336 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:55.651039 31218 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:19:55.739289 31337 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.642189 31224 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:55.738622 31327 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:19:55.749460 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.768610 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.764876 25786 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
W20260709 12:19:55.786087 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.786450 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.788520 25786 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:55.789851 30870 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.790365 31320 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.790720 31224 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.764907 25632 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:55.636314 31346 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.798010 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.800336 25786 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:55.802114 25782 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:55.802512 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.805208 25786 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:55.806018 25786 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:55.709340 31204 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.814291 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:55.816262 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:55.803536 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.817763 25782 raft_consensus.cc:2410] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:55.819470 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:55.821520 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.824790 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:55.825675 25637 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:55.827404 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.828295 25783 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:19:55.785046 31337 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:55.733557 31103 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.737881 31258 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.836918 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:55.837211 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.835721 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:55.839632 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:55.839651 25631 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:55.840016 25782 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:19:55.842777 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:55.823962 30866 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.846993 30866 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.849459 30866 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.818775 31084 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:55.850196 31084 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.850626 31084 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:55.854010 31103 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.820276 31281 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:55.854777 31336 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:19:55.855234 26337 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:55.855000 31281 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.856323 31258 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.857538 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.858857 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2743) took 1964 ms. Trace:
I20260709 12:19:55.860702 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:55.847394 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 992) took 1964 ms. Trace:
I20260709 12:19:55.861646 25712 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 15 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:55.861600 25784 rpcz_store.cc:276] 0709 12:19:53.883229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.453015 (+569786us) service_pool.cc:224] Handling call
0709 12:19:55.332817 (+879802us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:19:55.351021 (+ 18204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:55.351365 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.351395 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:55.367179 (+ 15784us) log.cc:844] Serialized 26 byte log entry
0709 12:19:55.576677 (+209498us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:55.576728 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.576806 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.845831 (+269025us) raft_consensus.cc:1685] finished
0709 12:19:55.845862 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.847228 (+  1366us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5400086bd0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:55.847339 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":665,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":56099584,"thread_start_us":209056,"threads_started":1,"wal-append.queue_time_us":353738}
I20260709 12:19:55.864238 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.860376 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.849787 26342 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:55.866047 25782 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:55.867226 25782 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:55.867321 25637 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:55.868026 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2742) took 1978 ms. Trace:
I20260709 12:19:55.737321 31335 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.868459 25786 rpcz_store.cc:276] 0709 12:19:53.890114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.827191 (+1937077us) service_pool.cc:224] Handling call
0709 12:19:55.867553 (+ 40362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.867732 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.867758 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:55.867798 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:55.867828 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.867877 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.867971 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.868027 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 999) took 1792 ms. Trace:
I20260709 12:19:55.695806 31299 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.861027 25783 rpcz_store.cc:276] 0709 12:19:53.894887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.829174 (+1934287us) service_pool.cc:224] Handling call
0709 12:19:55.858286 (+ 29112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.858466 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.858521 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:55.858564 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:55.858605 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.858695 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.858802 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.824201 31225 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.871493 25782 rpcz_store.cc:276] 0709 12:19:54.075526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.866730 (+1791204us) service_pool.cc:224] Handling call
0709 12:19:55.867039 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.867786 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.867973 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.790380 30870 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.857576 31281 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.832085 30856 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.878525 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:55.880542 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.887116 25629 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:55.884076 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2746) took 1953 ms. Trace:
I20260709 12:19:55.887116 25786 raft_consensus.cc:2376] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15. Re-sending same reply.
I20260709 12:19:55.884897 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.889772 25783 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:55.889780 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:55.890933 25629 raft_consensus.cc:2410] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:55.888455 25784 rpcz_store.cc:276] 0709 12:19:53.915207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.865105 (+1949898us) service_pool.cc:224] Handling call
0709 12:19:55.865385 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.868360 (+  2975us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:55.868400 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:55.868460 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:55.868511 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.868604 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.868743 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2619,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4515}
I20260709 12:19:55.894063 30856 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:55.884382 31340 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:19:55.884191 31344 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:55.896353 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:55.898893 25784 raft_consensus.cc:2376] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14. Re-sending same reply.
I20260709 12:19:55.899829 25786 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:55.900658 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2749) took 1387 ms. Trace:
W20260709 12:19:55.884058 31343 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:55.872326 31337 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:55.905721 30856 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.884829 31192 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.901983 25782 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:55.908720 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2747) took 1403 ms. Trace:
I20260709 12:19:55.909118 25782 rpcz_store.cc:276] 0709 12:19:54.505018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.888017 (+1382999us) service_pool.cc:224] Handling call
0709 12:19:55.897686 (+  9669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.908483 (+ 10797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.908665 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.900318 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.903980 25783 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:55.895658 31346 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.910765 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2210) took 1397 ms. Trace:
I20260709 12:19:55.901113 25786 rpcz_store.cc:276] 0709 12:19:54.512919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.888777 (+1375858us) service_pool.cc:224] Handling call
0709 12:19:55.899610 (+ 10833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.900367 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.900591 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.911504 25784 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:19:55.924492 25784 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:55.925834 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2214) took 1287 ms. Trace:
I20260709 12:19:55.911631 25783 rpcz_store.cc:276] 0709 12:19:54.513403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.892373 (+1378970us) service_pool.cc:224] Handling call
0709 12:19:55.903615 (+ 11242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.910530 (+  6915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.910711 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.927215 25786 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:55.926926 25784 rpcz_store.cc:276] 0709 12:19:54.638381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.923133 (+1284752us) service_pool.cc:224] Handling call
0709 12:19:55.923975 (+   842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.925562 (+  1587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.925773 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.940605 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1000) took 1306 ms. Trace:
I20260709 12:19:55.941200 25786 rpcz_store.cc:276] 0709 12:19:54.634604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.924101 (+1289497us) service_pool.cc:224] Handling call
0709 12:19:55.926863 (+  2762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.940265 (+ 13402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.940534 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.943030 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.944012 25786 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:55.945298 31299 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.947036 31179 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.950083 31250 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.952322 31335 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.931042 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.954582 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:55.955927 25781 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:19:55.957901 25781 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:55.954881 30870 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.958987 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1002) took 1289 ms. Trace:
I20260709 12:19:55.959561 25781 rpcz_store.cc:276] 0709 12:19:54.669887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.956856 (+1286969us) service_pool.cc:224] Handling call
0709 12:19:55.957272 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.958299 (+  1027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.958904 (+   605us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.961545 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2215) took 1277 ms. Trace:
I20260709 12:19:55.924880 31337 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:55.911906 31177 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:55.909350 31247 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:55.962088 25780 rpcz_store.cc:276] 0709 12:19:54.684422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.960487 (+1276065us) service_pool.cc:224] Handling call
0709 12:19:55.960844 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.961133 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.961168 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:55.961237 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:55.961298 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.961365 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.961477 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:55.971622 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 262448us
W20260709 12:19:55.972182 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 287234us
I20260709 12:19:55.945947 31292 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.976116 31292 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:55.980836 25783 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:55.981487 25780 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 12. Ops: [10.9-10.9]
I20260709 12:19:55.981866 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2216) took 1219 ms. Trace:
I20260709 12:19:55.982245 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2217) took 1204 ms. Trace:
I20260709 12:19:55.982524 25783 rpcz_store.cc:276] 0709 12:19:54.762896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.973823 (+1210927us) service_pool.cc:224] Handling call
0709 12:19:55.980569 (+  6746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.981518 (+   949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.981791 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:55.983166 25780 rpcz_store.cc:276] 0709 12:19:54.778128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.974599 (+1196471us) service_pool.cc:224] Handling call
0709 12:19:55.981153 (+  6554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:55.981984 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.982177 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:55.984850 25751 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 2.054s (SENT)
I20260709 12:19:55.987089 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:55.988441 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:55.910682 31192 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:55.992597 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:55.993773 25783 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:19:55.994889 25751 connection.cc:256] RPC call timeout handler was delayed by -1.0159725910002635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2764s for a response
I20260709 12:19:55.997816 25780 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 16. Ops: []
I20260709 12:19:55.998757 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2222) took 1013 ms. Trace:
I20260709 12:19:55.999200 25780 rpcz_store.cc:276] 0709 12:19:54.984938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.989392 (+1004454us) service_pool.cc:224] Handling call
0709 12:19:55.997629 (+  8237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.998388 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.998684 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.004206 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2219) took 1163 ms. Trace:
I20260709 12:19:56.004644 25783 rpcz_store.cc:276] 0709 12:19:54.841246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.988131 (+1146885us) service_pool.cc:224] Handling call
0709 12:19:55.988513 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:55.996089 (+  7576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.004146 (+  8057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.074607 31204 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:56.075686 25780 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:19:56.076507 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2752) took 1040 ms. Trace:
I20260709 12:19:56.076830 25780 rpcz_store.cc:276] 0709 12:19:55.035848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.074983 (+1039135us) service_pool.cc:224] Handling call
0709 12:19:56.075438 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:56.076231 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.076450 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.078379 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1252) took 1560 ms. Trace:
I20260709 12:19:56.078684 31354 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:56.078938 26338 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:56.078838 26335 rpcz_store.cc:276] 0709 12:19:54.517981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.518124 (+   143us) service_pool.cc:224] Handling call
0709 12:19:56.078196 (+1560072us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:56.078321 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1049978112}
W20260709 12:19:56.231477 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:56.293982 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.944021 31084 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.323125 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:56.328564 26335 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:56.012722 31177 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:56.381639 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:56.376554 25783 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:56.366783 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 256292us
I20260709 12:19:56.383086 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:56.352257 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.384666 25751 leader_election.cc:400] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:56.385692 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.386209 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:56.362655 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:56.394958 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 315369us
I20260709 12:19:56.328413 31152 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:56.367269 31227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.843s	user 0.032s	sys 0.031s
I20260709 12:19:56.366559 31318 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.396054 25603 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:56.415359 31330 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.414994 31225 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:56.464017 31227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.961s	user 0.033s	sys 0.031s
I20260709 12:19:55.996644 31351 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.494s
I20260709 12:19:56.528683 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:56.489244 31177 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:56.465935 31152 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:56.544075 31152 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:56.549057 25750 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:56.550213 31240 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:56.552243 31229 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:56.552454 31240 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.555060 31240 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.556121 31240 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:56.556979 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2831) took 1672 ms. Trace:
W20260709 12:19:56.557546 25709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2810) took 2711 ms (client timeout 2504 ms). Trace:
I20260709 12:19:56.560660 25708 rpcz_store.cc:276] 0709 12:19:54.884314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.000115 (+115801us) service_pool.cc:224] Handling call
0709 12:19:55.000419 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.336415 (+335996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.556748 (+1220333us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54000744d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:56.556922 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696308096}
W20260709 12:19:56.560868 25709 rpcz_store.cc:269] 0709 12:19:53.845644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.847527 (+  1883us) service_pool.cc:224] Handling call
0709 12:19:56.557286 (+2709759us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002238d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:56.557442 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2747653888}
I20260709 12:19:56.551017 31152 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:56.562176 31240 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.562688 31240 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:56.523981 31337 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:56.480755 31299 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:56.566715 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:19:56.588829 25679 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.23.217.126:36567, user_credentials={real_user=slave}} blocked reactor thread for 56196us
W20260709 12:19:56.593408 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:56.594182 26308 connection.cc:256] RPC call timeout handler was delayed by -1.098503755000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
I20260709 12:19:56.603842 26341 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:56.604032 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.596532 31216 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:56.612548 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 216600us
I20260709 12:19:56.526150 31330 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.606707 31152 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:19:56.633276 31152 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:56.604048 31337 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:56.497431 31318 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.661455 31268 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:56.665382 25783 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:19:56.666287 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2754) took 1603 ms. Trace:
I20260709 12:19:56.666720 25783 rpcz_store.cc:276] 0709 12:19:55.062805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.083778 (+1020973us) service_pool.cc:224] Handling call
0709 12:19:56.376405 (+292627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.665941 (+289536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.666212 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:56.674433 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.634189 31330 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.621544 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:19:56.678814 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 57369.2us
I20260709 12:19:56.682912 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:56.686419 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 300769us
W20260709 12:19:56.673563 31234 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 1.343s	user 0.016s	sys 0.000s
I20260709 12:19:56.661962 31268 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.661942 31337 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:19:56.685976 25677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.084s	sys 0.178s
I20260709 12:19:56.696744 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:56.692834 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:56.683756 25783 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:19:56.698462 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.700433 31337 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:56.701203 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:56.701498 31337 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:56.700038 25677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.085s	sys 0.178s
I20260709 12:19:56.699572 31268 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:56.702044 31337 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.703063 31239 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:56.704114 31237 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.704857 31318 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:56.704686 31237 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.706324 31084 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.701853 25783 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:56.708495 26335 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:56.607182 31299 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.708020 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.711318 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.712165 25629 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:56.634730 31216 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.722137 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.727366 26339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1262) took 1952 ms. Trace:
I20260709 12:19:56.727924 26339 rpcz_store.cc:276] 0709 12:19:54.775032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.804646 (+ 29614us) service_pool.cc:224] Handling call
0709 12:19:54.805053 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.133734 (+328681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.727020 (+1593286us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54004975d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:56.727285 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754463488}
I20260709 12:19:56.709178 31337 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:56.732389 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:56.734690 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.737498 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.739379 25783 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:56.739271 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:56.743661 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.744761 25633 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:56.745481 25629 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:56.747936 31216 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:56.748174 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.755544 31204 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:56.759995 31299 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.721657 31192 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.722536 31368 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.767045 31368 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.769253 31237 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:56.744148 31152 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:56.771224 31152 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:56.721462 31354 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:19:56.771461 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.744530 31239 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:56.774360 31239 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:56.777326 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:56.778404 25783 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:56.783262 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:56.735954 31055 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:56.790467 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.791193 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.792141 25637 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:56.780397 31225 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:56.795781 25630 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:19:56.785419 31292 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.806s	user 0.001s	sys 0.010s
I20260709 12:19:56.776479 31354 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:56.799989 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.801299 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.788772 31055 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.805433 25780 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:56.805090 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.815991 31208 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:56.822332 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.823572 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:56.826781 25783 raft_consensus.cc:2376] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18. Re-sending same reply.
I20260709 12:19:56.827036 25635 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:56.804407 31225 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:56.838526 26307 leader_election.cc:336] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.564s (SENT)
I20260709 12:19:56.816776 31055 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:56.858613 31370 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.858858 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:56.859325 31370 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.859813 25783 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:56.861169 25783 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:56.861795 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2233) took 1607 ms. Trace:
I20260709 12:19:56.816270 31179 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.862258 25783 rpcz_store.cc:276] 0709 12:19:55.254931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.860658 (+1605727us) service_pool.cc:224] Handling call
0709 12:19:56.860971 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.861535 (+   564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.861741 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.863057 31368 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.868296 31179 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.873796 31179 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.874155 31055 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:56.885504 25783 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:56.886445 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2236) took 1616 ms. Trace:
I20260709 12:19:56.886881 25783 rpcz_store.cc:276] 0709 12:19:55.270415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.864418 (+1594003us) service_pool.cc:224] Handling call
0709 12:19:56.885270 (+ 20852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.886117 (+   847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.886372 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.816488 31237 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:19:56.852872 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:56.891093 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 188371us
I20260709 12:19:56.861860 31370 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:56.874974 31225 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:56.879412 31353 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:56.819636 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.896999 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:56.899679 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:56.896021 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:56.903558 26339 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:56.900785 25633 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:56.900760 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:56.901412 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 179666us
I20260709 12:19:56.903841 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.905539 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:56.906270 25783 raft_consensus.cc:2376] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16. Re-sending same reply.
W20260709 12:19:56.906904 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 182956us
I20260709 12:19:56.908022 25783 raft_consensus.cc:1275] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:56.911300 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2237) took 1624 ms. Trace:
I20260709 12:19:56.909150 25635 raft_consensus.cc:2368] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:56.911612 25783 rpcz_store.cc:276] 0709 12:19:55.286844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.907150 (+1620306us) service_pool.cc:224] Handling call
0709 12:19:56.907630 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.911013 (+  3383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.911241 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:56.912703 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:56.920054 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:56.910600 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:19:56.921173 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 101662us
I20260709 12:19:56.922427 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:56.915330 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:56.925498 25783 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:19:56.923424 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.926921 25710 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:56.922892 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2750) took 2175 ms. Trace:
I20260709 12:19:56.929996 25677 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:56.930651 25631 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:56.927822 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:56.849812 31354 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:56.893534 31179 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:56.934377 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:56.934409 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:56.936511 25709 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:56.939898 25635 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:56.942040 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:56.928354 25781 rpcz_store.cc:276] 0709 12:19:54.747669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.960634 (+1212965us) service_pool.cc:224] Handling call
0709 12:19:55.961077 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.961356 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:56.438030 (+476674us) log.cc:844] Serialized 14 byte log entry
0709 12:19:56.922580 (+484550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:56.922637 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:56.922679 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.922743 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:56.922842 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6827,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"thread_start_us":960353,"threads_started":2}
I20260709 12:19:56.946933 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:56.947774 25635 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:56.941449 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:56.951192 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.952395 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.924384 31222 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:56.935673 31330 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:56.961102 25635 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:56.935474 31299 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:56.979249 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.963142 25711 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:56.973539 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.948156 31158 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:56.939647 31227 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:56.932184 31375 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:56.981117 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:56.929610 31231 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:19:56.949457 31213 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:56.991614 25706 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:56.934850 31216 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:19:56.999878 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.003409 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:56.917452 31353 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.984000 31229 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:56.961994 31354 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:57.013420 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.048592 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:57.015563 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.945218 31179 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:56.981758 31300 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.057578 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:56.958804 31318 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:56.986699 31158 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.032435 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.059636 31158 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:57.060324 31378 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:57.061225 31370 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:57.011337 31379 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:57.030733 31375 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:57.075330 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:57.054551 31369 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:57.070727 31239 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:57.071348 31380 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:19:57.072197 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:57.026036 31381 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:57.073557 31382 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:19:57.055876 31231 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:57.076133 25751 leader_election.cc:400] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:57.096076 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:57.096531 25751 leader_election.cc:403] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:19:57.096860 25709 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:57.091795 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.064244 31318 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.074628 31379 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.029983 31213 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:57.117884 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 144407us
W20260709 12:19:57.100227 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 153379us
I20260709 12:19:57.127118 25677 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:57.133913 31379 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:57.134811 31306 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:57.141817 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.143169 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.157018 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.157650 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.156952 25787 raft_consensus.cc:2410] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:57.144824 31330 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:57.067981 31300 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.064790 31240 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:57.168543 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2226) took 1997 ms (client timeout 2111 ms). Trace:
W20260709 12:19:57.249068 25787 rpcz_store.cc:269] 0709 12:19:55.171106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.141487 (+1970381us) service_pool.cc:224] Handling call
0709 12:19:57.168479 (+ 26992us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:57.180008 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52953.2us
I20260709 12:19:57.250298 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:57.250519 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:57.145483 31318 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:57.144491 31369 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:57.123751 31227 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:57.143177 31380 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:57.047886 25635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
W20260709 12:19:57.273268 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 167168us
I20260709 12:19:57.273919 25635 raft_consensus.cc:606] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:57.265645 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.159617 31190 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:57.270318 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:57.096076 31268 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:57.272753 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:57.280074 25708 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:57.262290 25786 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:57.169818 31370 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:57.276863 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:57.148574 31231 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.125s
I20260709 12:19:57.295120 25603 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:57.303049 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:57.304428 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.304611 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.311651 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.304987 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:57.312983 26341 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:57.263787 31369 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:57.315073 26343 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:57.322944 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:57.324115 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:57.324790 26337 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:57.325356 26343 raft_consensus.cc:2410] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:19:57.315642 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 209254us
I20260709 12:19:57.328478 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.332193 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:57.335983 31383 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:19:57.266812 31341 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:19:57.339437 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.361157 25630 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:57.364359 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.368881 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:57.297358 31392 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.342084 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.356395 26341 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:57.334223 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:57.283578 31239 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:19:57.326761 31299 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:57.368247 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:57.379980 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.372572 26340 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:19:57.369750 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:57.291921 31387 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:57.359772 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 925) took 2610 ms. Trace:
I20260709 12:19:57.380841 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.327911 31015 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:57.391450 26344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1264) took 2447 ms (client timeout 1542 ms). Trace:
W20260709 12:19:57.392052 26344 rpcz_store.cc:269] 0709 12:19:54.924598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.926602 (+  2004us) service_pool.cc:224] Handling call
0709 12:19:57.372029 (+2445427us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54000a5d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:57.372185 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910778752}
I20260709 12:19:57.392521 26307 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.389664 25709 raft_consensus.cc:2410] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:19:57.381433 25705 rpcz_store.cc:276] 0709 12:19:54.718742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.752194 (+ 33452us) service_pool.cc:224] Handling call
0709 12:19:54.752597 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.872163 (+119566us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:54.872224 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.872850 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:19:55.094341 (+221491us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:55.094396 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.311637 (+2217241us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b5400104854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:57.311708 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:57.311775 (+    67us) raft_consensus.cc:1685] finished
0709 12:19:57.311795 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.329054 (+ 17259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227160320,"thread_start_us":221114,"threads_started":1,"wal-append.queue_time_us":311738,"wal-append.run_cpu_time_us":1699,"wal-append.run_wall_time_us":1028839}
I20260709 12:19:57.394048 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:57.394616 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:57.299140 31152 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:19:57.340672 31237 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.330377 31395 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:57.411811 25708 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:57.287962 31190 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.415341 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:57.415424 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 227785us
I20260709 12:19:57.391076 31015 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.413157 31301 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:57.426836 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.440253 25630 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:57.440909 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.440910 25630 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:19:57.430971 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 135961us
I20260709 12:19:57.391898 31299 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.413769 31387 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:19:57.462860 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 70428.4us
I20260709 12:19:57.464807 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:57.421217 31353 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:57.472108 31389 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:19:57.440724 31015 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:57.432228 31231 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.429996 31179 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.414629 31237 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.444828 31301 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.445026 31299 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:57.440450 31401 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.589493 31401 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:57.482412 31229 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.591142 31229 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.592223 31401 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:57.593618 31229 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.427156 31234 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:19:57.416648 31396 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:19:57.594973 31396 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:57.594864 31379 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.468022 31385 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:57.599079 31208 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.000s
I20260709 12:19:57.599148 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 960) took 2840 ms. Trace:
I20260709 12:19:57.564524 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2757) took 2328 ms. Trace:
I20260709 12:19:57.454092 31366 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:19:57.600945 25780 rpcz_store.cc:276] 0709 12:19:55.236639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.185949 (+949310us) service_pool.cc:224] Handling call
0709 12:19:56.373161 (+187212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.814788 (+441627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.564279 (+749491us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540030af54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:57.564462 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993166080}
I20260709 12:19:57.462371 31402 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:19:57.600147 25708 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:57.603755 31402 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:57.413936 31227 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.433023 31398 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 12:19:57.420706 31369 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:57.609203 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 142420us
W20260709 12:19:57.558805 25711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 933) took 2218 ms (client timeout 2256 ms). Trace:
W20260709 12:19:57.554328 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2240) took 2266 ms (client timeout 2011 ms). Trace:
W20260709 12:19:57.609848 25711 rpcz_store.cc:269] 0709 12:19:55.340186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.345013 (+  4827us) service_pool.cc:224] Handling call
0709 12:19:57.558585 (+2213572us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400105254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:57.558726 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3708427008}
W20260709 12:19:57.610222 25778 rpcz_store.cc:269] 0709 12:19:55.287807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.554033 (+2266226us) service_pool.cc:211] Skipping call since client already timed out
0709 12:19:57.554201 (+   168us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:19:57.536378 31387 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:57.600579 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 1007) took 2214 ms (client timeout 1572 ms). Trace:
I20260709 12:19:57.493036 31407 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:57.557365 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2376) took 1744 ms. Trace:
W20260709 12:19:57.613010 25779 rpcz_store.cc:269] 0709 12:19:55.386000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.600392 (+2214392us) service_pool.cc:211] Skipping call since client already timed out
0709 12:19:57.600497 (+   105us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:19:57.569131 31237 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:57.613730 31407 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:57.569681 31268 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:57.543179 25630 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:57.613425 25632 rpcz_store.cc:276] 0709 12:19:55.812542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.823467 (+ 10925us) service_pool.cc:224] Handling call
0709 12:19:55.823807 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.906692 (+ 82885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.906745 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:55.906810 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:55.906868 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.557036 (+1650168us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b54002ac6d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:57.557105 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.557287 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189964160}
I20260709 12:19:57.562067 31190 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.599948 25713 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:57.595494 31292 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:57.616091 31247 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.616281 31292 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:19:57.616766 31375 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.003s
I20260709 12:19:57.562323 31254 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:57.617161 31375 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:57.616781 31247 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:57.496623 31388 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:19:57.494717 31240 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.524662 31231 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.599675 26338 rpcz_store.cc:276] 0709 12:19:54.758358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.760099 (+  1741us) service_pool.cc:224] Handling call
0709 12:19:56.078391 (+1318292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.079859 (+  1468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.598907 (+1519048us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400563450. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:57.599062 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010152704}
I20260709 12:19:57.619894 31318 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.516731 31239 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.392511 31392 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:57.493376 31337 consensus_meta.cc:287] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.001s
I20260709 12:19:57.526178 31301 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:57.645053 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.581140 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3042) took 1921 ms. Trace:
W20260709 12:19:57.581372 31405 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:19:57.641698 31318 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:57.595485 31396 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:57.648072 31396 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.649406 31396 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:57.505059 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:57.651345 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 146398us
I20260709 12:19:57.652807 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.654390 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:57.655086 25638 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:57.655161 31393 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:57.656050 31393 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:57.656672 31393 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:57.657120 31239 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.660969 31239 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:57.662063 25521 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 60051e5723b2429582407076a003096c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0ae979356d694a09be4b0cd976ea354a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:19:57.664481 31015 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:57.665652 25629 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:57.598050 31300 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.668268 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:57.597684 31299 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.598834 31385 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.392719 31330 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.519245 31376 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:57.682204 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:57.682564 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:57.683197 26336 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:57.554322 31158 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.617667 31254 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.526839 31213 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.620568 31231 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.620785 31055 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:57.524268 31391 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.622095 31204 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:57.614656 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1084) took 2262 ms. Trace:
I20260709 12:19:57.682567 31216 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:57.630517 31378 raft_consensus.cc:993] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:19:57.684315 31015 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:57.631264 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:57.635164 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.617249 31292 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.639673 26337 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:57.639348 31367 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:57.681516 31354 consensus_queue.cc:1059] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:19:57.643102 31275 raft_consensus.cc:3055] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:57.643908 31247 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:57.643666 31337 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.587249 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 972) took 1250 ms. Trace:
I20260709 12:19:57.646260 25704 rpcz_store.cc:276] 0709 12:19:55.659519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.697675 (+ 38156us) service_pool.cc:224] Handling call
0709 12:19:55.697973 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:55.859473 (+161500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:55.859515 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:55.859560 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:55.859595 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.580862 (+1721267us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54003ab454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:57.580915 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.581069 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370955648}
I20260709 12:19:57.669924 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.758864 25630 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:57.671646 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.690356 31364 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 12:19:57.762545 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 90961us
I20260709 12:19:57.678973 25638 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
W20260709 12:19:57.685362 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.690048 26339 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:57.760622 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:19:57.684463 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 345166us
I20260709 12:19:57.769515 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.682870 31233 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:57.630266 31381 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.526s	user 0.003s	sys 0.004s
I20260709 12:19:57.730512 26340 rpcz_store.cc:276] 0709 12:19:55.352338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.352472 (+   134us) service_pool.cc:224] Handling call
0709 12:19:57.372312 (+2019840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:57.393677 (+ 21365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.614424 (+220747us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54000a5d50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:57.614599 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4602222336}
I20260709 12:19:57.733235 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:57.747474 31346 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:57.624538 31407 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.742216 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:57.701488 31254 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:57.744297 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 113143us
I20260709 12:19:57.641788 31392 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.758646 26335 rpcz_store.cc:276] 0709 12:19:56.336755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.381291 (+ 44536us) service_pool.cc:224] Handling call
0709 12:19:57.587041 (+1205750us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54005bf9d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:57.587179 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749892864}
W20260709 12:19:57.677103 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.766225 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:19:57.691030 31376 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:57.748492 31055 raft_consensus.cc:1081] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:19:57.690224 31323 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:19:57.779397 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.775887 31231 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:57.796299 25631 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:57.820055 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.821241 25778 raft_consensus.cc:2410] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:57.758910 31275 raft_consensus.cc:740] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:57.763616 31292 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:57.811882 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.816166 31299 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:57.741074 26336 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:19:57.814625 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.816340 31376 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:57.816738 31375 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:57.801177 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:57.796110 31318 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:57.805871 31378 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:57.836748 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.822587 25752 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.822945 25637 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:19:57.790298 31179 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:57.825344 31268 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:57.826180 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.800361 31233 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:57.805665 31386 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:57.742574 31391 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:57.812577 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 46448.7us
I20260709 12:19:57.803921 31240 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.799602 31381 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:57.831387 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:57.833177 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.834766 25787 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:57.948148 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.809931 31401 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:57.835402 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.954927 25783 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:19:57.956271 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.850616 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:57.828697 31380 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.543s	user 0.005s	sys 0.005s
I20260709 12:19:57.851819 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:57.956979 25783 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:19:57.846977 25778 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:19:57.859277 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.860865 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.963910 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.830036 31346 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:57.827432 31177 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:57.853369 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:57.911821 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.916522 25779 raft_consensus.cc:2410] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:19:57.916863 25711 raft_consensus.cc:1240] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:57.917309 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.917435 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:57.826893 31351 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:57.845388 31318 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.998886 25713 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:57.950935 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:57.946758 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:57.851671 31378 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:19:57.962697 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.964008 25786 raft_consensus.cc:2376] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16. Re-sending same reply.
W20260709 12:19:57.990322 25601 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.988s (SENT)
I20260709 12:19:57.965168 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.986548 25780 raft_consensus.cc:2376] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15. Re-sending same reply.
I20260709 12:19:57.928486 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.013263 26338 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:19:58.016989 25781 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:19:57.929301 31224 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:57.965184 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:57.991688 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.014305 25784 raft_consensus.cc:2368] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
W20260709 12:19:57.873561 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 403884us
I20260709 12:19:57.967955 31401 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.015348 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:58.023440 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 171675us
I20260709 12:19:57.928596 31239 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:57.954744 25707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:19:58.020263 25778 raft_consensus.cc:2468] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:58.019981 25706 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:19:57.994024 25633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.005s	sys 0.022s
I20260709 12:19:57.918402 31375 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:58.054236 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:57.953403 31398 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:58.032799 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.037268 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.061769 25780 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:58.062630 25780 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:58.038094 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.063403 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.063906 25779 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:58.065482 25783 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:57.987124 31369 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:58.035801 25707 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:58.076910 25629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:19:58.077231 25707 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:58.033633 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:58.078398 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44907us
I20260709 12:19:58.079859 25676 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:57.928382 31055 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.103s
W20260709 12:19:58.022297 31225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.016s	sys 0.017s
I20260709 12:19:58.048179 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.031008 31419 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:58.049227 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:19:58.046823 25633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.005s	sys 0.024s
I20260709 12:19:58.015213 31318 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:58.099237 31428 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:19:58.086760 31421 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.101078 31421 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.057746 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.102618 25786 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:57.965869 31400 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:57.994303 31380 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:19:58.060926 31158 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.107317 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.077545 25629 raft_consensus.cc:606] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:58.121348 25629 raft_consensus.cc:614] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:58.082412 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.117473 25706 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:58.036334 31381 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:19:58.048266 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 268205us
I20260709 12:19:57.961699 31240 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.090234 25780 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:58.093017 25779 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:19:58.096835 31416 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:19:58.097366 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.141831 25712 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
W20260709 12:19:58.093287 31225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.018s	sys 0.017s
I20260709 12:19:58.093313 31335 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.105221 25781 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:58.014933 31301 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.068104 31385 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.075385 31359 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.087272 31299 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:58.090859 31247 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:58.123273 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:58.088119 31239 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:58.048281 31224 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.162832 25707 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.129278 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:58.082166 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.085196 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:19:58.173422 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79540.8us
I20260709 12:19:58.173727 25711 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:58.174158 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 89043.7us
I20260709 12:19:58.175606 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:58.088846 25713 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:58.137792 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.176805 25708 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:58.142725 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 152435us
I20260709 12:19:58.177275 25779 raft_consensus.cc:2393] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:58.143111 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.107998 31177 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.182943 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.152001 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.168931 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.178480 25786 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:58.187619 25707 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.094695 31251 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:19:58.173317 25704 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:19:58.087936 31398 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.180928 25676 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:58.181506 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:58.151407 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.186200 25780 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:58.152958 31027 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:58.192090 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.106703 31419 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.047595 31275 consensus_queue.cc:260] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.087857 31233 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.104086 31346 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.201707 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.119350 31400 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.238754 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.176132 31231 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:58.220157 31415 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:19:58.220629 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.221138 31301 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.195940 31251 consensus_queue.cc:1243] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:19:58.222358 31413 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:19:58.221807 31385 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:58.223090 31084 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:58.224529 25603 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:58.225067 31270 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.225561 31406 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:58.226012 31225 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.240748 31299 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:58.227305 31421 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.227895 31224 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:58.186192 31418 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:58.229364 31419 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:58.231814 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:58.233042 31391 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.221774 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.234447 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:19:58.297742 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63417.7us
I20260709 12:19:58.235623 25787 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:19:58.236155 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:58.221065 31424 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:19:58.233713 31216 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:58.243571 31425 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:19:58.244597 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.186307 31392 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:58.119480 31378 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.320969 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.248528 25786 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:19:58.249048 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.262135 25779 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:19:58.272089 31240 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:58.276976 31396 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.627s	user 0.006s	sys 0.006s
I20260709 12:19:58.168103 31359 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.278163 26344 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:19:58.279106 31400 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:19:58.283263 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 83064.5us
I20260709 12:19:58.284978 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.159075 31335 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:58.289410 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50742.3us
I20260709 12:19:58.344481 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:58.345505 25704 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:58.283955 31227 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:58.189850 31177 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.317385 31216 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:58.240698 31323 consensus_queue.cc:1059] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:19:58.299582 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:58.280944 31385 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:58.370858 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.323671 31392 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:58.244047 31402 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:19:58.247174 26335 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:58.288890 31301 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:58.317803 31027 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:19:58.322033 31364 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:19:58.322186 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.328636 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.331331 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.383230 26341 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:19:58.335786 31359 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:58.338372 31435 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:58.297021 31225 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.291779 31270 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.345676 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.301987 31368 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:58.168412 31375 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.360725 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:58.302328 31418 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:58.305126 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2839) took 1224 ms. Trace:
I20260709 12:19:58.287955 31196 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:19:58.390950 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 45339.3us
I20260709 12:19:58.389712 25709 rpcz_store.cc:276] 0709 12:19:57.080484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.414208 (+333724us) service_pool.cc:224] Handling call
0709 12:19:57.414586 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:57.414941 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:57.414976 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:57.423379 (+  8403us) log.cc:844] Serialized 26 byte log entry
0709 12:19:57.685334 (+261955us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:57.685391 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.685482 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.304815 (+619333us) raft_consensus.cc:1685] finished
0709 12:19:58.304844 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.305046 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9032,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":3793,"thread_start_us":232385,"threads_started":1,"wal-append.queue_time_us":499946}
I20260709 12:19:58.322218 31351 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:19:58.384634 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:58.373637 31385 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:58.378687 31392 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:58.366199 31227 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.395519 31418 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:58.402123 31196 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:58.425366 25780 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:19:58.402446 31015 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:58.407261 31177 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.397500 31084 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:58.410374 31270 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.387776 31368 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:58.397097 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:58.411851 31299 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:58.426038 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.418083 31247 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:58.436421 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:58.436926 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 137378us
I20260709 12:19:58.419301 31387 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:58.462074 25786 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:58.434085 31379 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.465929 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:58.432484 31015 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:58.472693 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:58.474769 31402 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:58.463079 31368 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:58.476470 25707 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:58.482725 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.471040 31379 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:58.470402 31084 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:58.489022 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:58.489477 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 100527us
I20260709 12:19:58.491492 25786 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
W20260709 12:19:58.494216 31439 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:19:58.491822 31196 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.508397 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.491276 31084 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:58.487427 31177 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:19:58.536104 31177 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:58.536473 31177 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:58.537470 31177 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.538796 31398 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.502498 31015 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.476156 31318 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.506717 31240 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:58.538801 26343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:58.544260 26343 raft_consensus.cc:606] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:58.544562 26343 raft_consensus.cc:614] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:58.527963 25709 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:58.535410 31084 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:58.547204 25783 raft_consensus.cc:2410] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 12:19:58.548864 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 151905us
I20260709 12:19:58.560361 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:19:58.562883 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.557266 31015 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:58.565719 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.567283 31447 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.572993 25710 raft_consensus.cc:1240] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:19:58.572899 31447 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.575196 31447 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.576083 31335 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.563213 31379 consensus_queue.cc:1048] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:19:58.576996 31216 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.558525 31330 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:58.577996 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.578629 25632 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:58.579994 31227 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.580277 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.581213 25784 raft_consensus.cc:2410] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:19:58.582103 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.561863 31177 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:58.583274 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.587172 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.585448 25784 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:19:58.549316 31240 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.593591 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:19:58.598378 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.552129 31392 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.594969 25780 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:58.609302 25780 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:58.606891 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.611505 25784 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:19:58.562800 31386 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:19:58.601940 31254 consensus_meta.cc:287] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.003s
I20260709 12:19:58.588709 31419 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.587214 31449 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:58.623179 26343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:19:58.629120 26343 raft_consensus.cc:606] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:19:58.629470 26343 raft_consensus.cc:614] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:58.626572 25706 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:58.617947 31369 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:19:58.633304 31292 consensus_meta.cc:287] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.798s	user 0.008s	sys 0.005s
W20260709 12:19:58.641491 31452 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:19:58.667137 31318 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.674851 31216 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.676828 31392 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.679483 26343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
W20260709 12:19:58.685070 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.689719 31240 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:58.690806 25782 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:58.692732 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.693650 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.695865 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.696727 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:58.697889 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.698364 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.699035 25638 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:19:58.625520 31254 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.701916 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:19:58.701980 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74950.3us
I20260709 12:19:58.703127 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.713251 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.714076 25705 raft_consensus.cc:2410] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:19:58.714762 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.715399 25705 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:58.732017 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.735901 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:58.631985 31419 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.634539 31330 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
W20260709 12:19:58.702010 31437 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:19:58.761407 26337 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:58.646926 31369 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.763706 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:58.764600 26343 raft_consensus.cc:606] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:58.765223 26343 raft_consensus.cc:614] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:58.733474 31225 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.737251 31292 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:58.773144 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:58.801244 25637 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:58.775992 31392 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:58.757032 31421 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:19:58.765461 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:58.806746 25705 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:58.808835 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.812733 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:58.811774 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46408.8us
I20260709 12:19:58.817351 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:58.807153 31292 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.821090 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.823545 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:58.831835 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:58.833197 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.834596 25780 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 19 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:19:58.836091 25676 leader_election.cc:400] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:19:58.842145 25676 leader_election.cc:403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:58.841740 25705 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:19:58.814990 31459 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:58.844187 31369 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:58.845659 31254 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:58.839263 31386 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:58.835516 31299 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:58.847709 31459 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.839648 31449 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:58.811718 31368 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.820155 31292 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:58.872682 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:19:58.875363 31460 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:19:58.879487 31419 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.842974 31084 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.870460 25711 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:19:58.888682 31456 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:19:58.889756 31455 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:19:58.940855 31401 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:58.890470 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:58.890941 31027 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:58.948531 25781 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:58.948676 31254 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.949182 31254 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.834730 31461 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:58.950677 31461 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:58.951078 31461 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:58.860198 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.954025 25783 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:19:58.955122 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.956183 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.957168 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.957707 25783 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
W20260709 12:19:58.959450 31418 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.006s
I20260709 12:19:58.959880 31418 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:58.948558 31027 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:58.962527 25709 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:58.967422 31027 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:58.968511 31027 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:58.933300 31386 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:58.970703 31386 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:58.934885 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 98910.3us
I20260709 12:19:58.835927 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:58.981619 25780 raft_consensus.cc:2393] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:58.967757 25712 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:58.958447 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.987756 25783 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
W20260709 12:19:58.990398 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:58.936012 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.980077 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:59.069936 25783 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:59.072957 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1252) took 1910 ms. Trace:
I20260709 12:19:59.073519 25633 rpcz_store.cc:276] 0709 12:19:57.162122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.162271 (+   149us) service_pool.cc:224] Handling call
0709 12:19:57.254780 (+ 92509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:57.463821 (+209041us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:57.463913 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:57.465478 (+  1565us) log.cc:844] Serialized 26 byte log entry
0709 12:19:58.094250 (+628772us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:58.094298 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.943656 (+849358us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b54001e91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:58.943712 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.943759 (+    47us) raft_consensus.cc:1685] finished
0709 12:19:58.943773 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:59.072869 (+129096us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29030,"raft.queue_time_us":35601,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11901,"spinlock_wait_cycles":116402944,"thread_start_us":785517,"threads_started":2,"wal-append.queue_time_us":790503}
W20260709 12:19:59.080801 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 100790us
I20260709 12:19:59.081830 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:19:59.081740 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:58.933701 31459 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:59.080164 31421 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:19:59.156029 31213 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:59.161556 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.160722 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 79002.5us
I20260709 12:19:59.164722 25781 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:19:59.167125 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:59.160737 31213 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:59.175477 31213 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.176524 31213 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:59.186931 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.167834 31239 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:59.189034 31239 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:59.188505 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1248) took 2041 ms. Trace:
I20260709 12:19:59.190038 31239 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.157186 31421 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:59.190677 25631 rpcz_store.cc:276] 0709 12:19:57.147004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.147206 (+   202us) service_pool.cc:224] Handling call
0709 12:19:57.147479 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:57.826887 (+679408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.188231 (+1361344us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b5400364554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:59.188437 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491671296}
I20260709 12:19:59.195114 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:59.190438 31318 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:59.198140 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 984) took 1072 ms. Trace:
I20260709 12:19:59.161880 31254 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.161720 31449 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:59.196702 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:59.200387 26344 rpcz_store.cc:276] 0709 12:19:58.125970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.126258 (+   288us) service_pool.cc:224] Handling call
0709 12:19:59.197943 (+1071685us) spinlock_profiling.cc:243] Waited 83.5 ms on lock 0x7b54006caf54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:59.198074 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191985664}
I20260709 12:19:59.202395 31239 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:59.203179 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:59.206928 31447 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:19:59.207103 31401 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.194257 31421 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.199152 31472 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:59.211098 31318 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.233346 31318 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.234824 31418 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.211894 31299 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:59.218334 31401 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.215229 31254 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.223856 31292 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.224138 31421 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:59.227408 31466 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:19:59.255827 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1094) took 1096 ms. Trace:
I20260709 12:19:59.225615 31472 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:59.256672 31472 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:59.256462 26337 rpcz_store.cc:276] 0709 12:19:58.159937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.321916 (+161979us) service_pool.cc:224] Handling call
0709 12:19:59.255558 (+933642us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b540063fe54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:59.255748 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870650240}
I20260709 12:19:59.259392 26342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 988) took 1132 ms. Trace:
I20260709 12:19:59.261786 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:59.259949 26342 rpcz_store.cc:276] 0709 12:19:58.127149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.127960 (+   811us) service_pool.cc:224] Handling call
0709 12:19:58.484638 (+356678us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:58.488651 (+  4013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:58.488694 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:58.489273 (+   579us) log.cc:844] Serialized 52 byte log entry
0709 12:19:58.559813 (+ 70540us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:58.559863 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.559937 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.801254 (+241317us) raft_consensus.cc:1685] finished
0709 12:19:58.801278 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:59.259051 (+457773us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54005a0fd0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:59.259313 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1970,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":4220,"spinlock_wait_cycles":819427072,"thread_start_us":50662,"threads_started":1,"wal-append.queue_time_us":60458}
I20260709 12:19:59.226997 31449 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:59.267884 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:59.269896 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:19:59.266252 31385 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.837s	user 0.007s	sys 0.007s
I20260709 12:19:59.274142 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:59.276144 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.276681 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.272876 31177 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:59.219003 31447 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:59.265506 31368 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.288951 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.292743 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.294595 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.294790 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:59.296084 25711 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:19:59.299312 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:59.261941 31254 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.298452 31225 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:59.287042 31231 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:59.279289 31449 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:59.287223 31177 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.309856 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.288082 31275 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:59.286265 31292 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.312019 31225 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:59.316581 31461 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:59.322342 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.285950 31385 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:59.318710 31447 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.318810 31177 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:59.309861 31231 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:59.299707 31386 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:59.336527 31392 consensus_meta.cc:287] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.001s
I20260709 12:19:59.339270 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.337911 31337 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:19:59.348428 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2759) took 1493 ms (client timeout 1870 ms). Trace:
W20260709 12:19:59.348860 25785 rpcz_store.cc:269] 0709 12:19:57.855316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.959281 (+103965us) service_pool.cc:224] Handling call
0709 12:19:58.420067 (+460786us) spinlock_profiling.cc:243] Waited 945 us on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:19:59.339098 (+919031us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ae9c 00007f29b286aa5b 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba9550a0 00007f29ba954984 00007f29b3ecf6d4 00007f29b3ecf662
0709 12:19:59.348360 (+  9262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15301888}
I20260709 12:19:59.349272 31461 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.349861 31216 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:59.350131 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.350306 31216 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:59.350768 25785 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:19:59.350975 31216 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.351940 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.352243 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3081) took 1007 ms. Trace:
I20260709 12:19:59.352576 25785 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:59.343007 31231 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:59.360005 25785 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:19:59.347178 31392 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:59.352784 25706 rpcz_store.cc:276] 0709 12:19:58.344628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.346204 (+  1576us) service_pool.cc:224] Handling call
0709 12:19:58.368564 (+ 22360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.630070 (+261506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.351977 (+721907us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54003c8cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:59.352175 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":592819584}
I20260709 12:19:59.363010 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.364457 25785 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:19:59.360301 31386 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.321875 31401 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.365900 25713 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:59.361332 31084 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.375905 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.368682 31391 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.376025 31401 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.397982 31254 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:59.381383 31318 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:59.401871 31177 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.361446 31449 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:19:59.437608 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 175923us
I20260709 12:19:59.395529 31391 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:59.441253 25787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2266) took 1605 ms (client timeout 1751 ms). Trace:
W20260709 12:19:59.441740 25787 rpcz_store.cc:269] 0709 12:19:57.836306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.947789 (+111483us) service_pool.cc:224] Handling call
0709 12:19:59.441030 (+1493241us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400099f54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:59.441185 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468567424}
I20260709 12:19:59.445009 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2851) took 1359 ms. Trace:
I20260709 12:19:59.423254 31401 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.430909 31216 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:19:59.456866 25782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2261) took 1625 ms (client timeout 1941 ms). Trace:
I20260709 12:19:59.456983 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:59.457427 25782 rpcz_store.cc:269] 0709 12:19:57.831921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.832038 (+   117us) service_pool.cc:224] Handling call
0709 12:19:59.456701 (+1624663us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54001252d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:59.456812 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1671909120}
I20260709 12:19:59.445560 25708 rpcz_store.cc:276] 0709 12:19:58.085685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.122450 (+ 36765us) service_pool.cc:224] Handling call
0709 12:19:58.176628 (+ 54178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.177189 (+   561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.444759 (+1267570us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b5400105c50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:59.444929 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123816704}
I20260709 12:19:59.448412 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:59.429646 31318 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.450011 31177 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.453372 31391 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:59.455021 31385 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.455694 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3080) took 1111 ms. Trace:
I20260709 12:19:59.456344 25785 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:59.467653 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.458186 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.458426 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.461818 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.466898 25713 rpcz_store.cc:276] 0709 12:19:58.344352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.344973 (+   621us) service_pool.cc:224] Handling call
0709 12:19:58.355202 (+ 10229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:59.370310 (+1015108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.455418 (+ 85108us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001bf654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:59.455627 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323953536}
I20260709 12:19:59.469321 25787 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
W20260709 12:19:59.464154 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 165035us
I20260709 12:19:59.468729 26342 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:59.470592 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.471408 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:59.470044 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44674.9us
I20260709 12:19:59.473150 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.473564 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.474298 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:19:59.475417 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.475454 25781 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:19:59.472988 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:59.456867 31401 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.458532 31216 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.474370 26342 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:19:59.480708 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.483912 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.480892 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.458182 31421 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.485424 26337 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:59.488952 25785 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:59.458617 31231 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.493230 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:59.474547 31479 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:19:59.500509 31479 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.502722 31479 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.496083 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.496841 26337 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:59.493036 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.507639 25708 raft_consensus.cc:2376] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16. Re-sending same reply.
I20260709 12:19:59.464641 31472 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:59.497004 25750 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:59.497586 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.517549 26342 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:19:59.512923 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:59.515529 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:59.498276 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.522154 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.523850 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:19:59.525542 25750 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:59.527311 25710 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:59.528561 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:19:59.530835 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:59.531896 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:19:59.533433 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.534384 25711 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:59.535840 25750 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:59.521087 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:59.484913 31385 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:19:59.538901 31225 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:59.562290 31472 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.551471 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.506160 31459 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.581305 31459 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.537073 31177 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:59.570070 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.582522 31177 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.537343 31239 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:59.582943 31177 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:59.554525 31463 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.707s	user 0.019s	sys 0.029s
I20260709 12:19:59.590961 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.591670 25710 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:19:59.564218 31318 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.575914 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.599709 25787 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:19:59.601052 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.577163 31254 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:59.527194 31481 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:19:59.537050 31485 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:59.581528 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1023) took 1462 ms. Trace:
I20260709 12:19:59.582016 25706 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:19:59.570474 31421 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.539907 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:19:59.625615 25711 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 9.
W20260709 12:19:59.585027 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64010.7us
I20260709 12:19:59.627768 26307 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:59.630811 31318 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:59.631430 31318 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.631909 31318 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:59.636747 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.638263 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.639014 25631 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:59.639189 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3076) took 1296 ms. Trace:
I20260709 12:19:59.639093 31315 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:19:59.586411 25712 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:59.587399 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 53813.8us
I20260709 12:19:59.642200 25712 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:59.643272 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:19:59.590535 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:19:59.615422 25784 rpcz_store.cc:276] 0709 12:19:58.118795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.613786 (+494991us) service_pool.cc:224] Handling call
0709 12:19:58.624872 (+ 11086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:58.797638 (+172766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:58.797681 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:58.802206 (+  4525us) log.cc:844] Serialized 26 byte log entry
0709 12:19:58.838998 (+ 36792us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:58.839042 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.581146 (+742104us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001a77d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:59.581199 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:59.581247 (+    48us) raft_consensus.cc:1685] finished
0709 12:19:59.581262 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:59.581471 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":3204,"spinlock_wait_cycles":339584384,"thread_start_us":19093,"threads_started":1,"wal-append.queue_time_us":278721}
W20260709 12:19:59.644579 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.617453 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.645609 25787 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.621003 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 939) took 1048 ms. Trace:
I20260709 12:19:59.638197 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.507165 31401 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:59.561468 31231 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.639645 25707 rpcz_store.cc:276] 0709 12:19:58.342982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.343188 (+   206us) service_pool.cc:224] Handling call
0709 12:19:58.343659 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:58.478652 (+134993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.638959 (+1160307us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400073fd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:19:59.639130 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304957824}
W20260709 12:19:59.651049 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60658.4us
I20260709 12:19:59.643136 25712 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.652283 25712 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:59.646181 25704 rpcz_store.cc:276] 0709 12:19:58.595175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.595365 (+   190us) service_pool.cc:224] Handling call
0709 12:19:58.596460 (+  1095us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:58.596962 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:58.618050 (+ 21088us) log.cc:844] Serialized 14 byte log entry
0709 12:19:58.794805 (+176755us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:58.795461 (+   656us) log.cc:844] Serialized 52 byte log entry
0709 12:19:58.795620 (+   159us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:58.801266 (+  5646us) log.cc:844] Serialized 14 byte log entry
0709 12:19:58.802192 (+   926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.802283 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:59.362509 (+560226us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b180034cae0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:19:59.553723 (+191214us) spinlock_profiling.cc:243] Waited 9.46 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:19:59.620673 (+ 66950us) raft_consensus.cc:1685] finished
0709 12:19:59.620708 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:59.620917 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38507,"prepare.queue_time_us":24591,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":43066,"raft.run_cpu_time_us":472,"raft.run_wall_time_us":23632,"spinlock_wait_cycles":28834944,"thread_start_us":138841,"threads_started":2,"wal-append.queue_time_us":208374}
I20260709 12:19:59.613615 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1025) took 1491 ms. Trace:
I20260709 12:19:59.660673 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.665468 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.667471 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.670403 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.671276 25629 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:59.677492 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.592545 31476 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:59.690125 25633 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:59.689079 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:19:59.690429 26342 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:59.691002 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.594606 31292 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:59.594105 31179 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.688917 25630 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:59.611217 31254 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.690330 25638 raft_consensus.cc:2393] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:59.639859 31315 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.759742 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.640877 31419 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:59.724745 25631 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:59.703883 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:19:59.712292 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:59.733273 25787 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:19:59.725769 25630 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:59.783142 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:59.783970 25638 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:19:59.784548 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:19:59.780627 25635 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:19:59.786293 25632 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:59.787209 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:19:59.790956 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.791702 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:19:59.794118 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 90347.2us
I20260709 12:19:59.794771 26339 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:59.795027 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.690141 31239 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:59.692999 31346 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:59.694918 31485 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:59.767864 31482 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:59.799129 31447 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:19:59.805290 31275 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:59.806473 31447 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:59.807729 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.806684 31368 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:59.808508 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.699059 31292 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.759392 31401 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:59.716899 31015 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.779269 31254 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:59.811717 25632 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:59.710140 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 157879us
I20260709 12:19:59.821391 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:19:59.821727 25712 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:19:59.822054 25704 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:59.824462 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.816802 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.703711 25634 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:59.825894 25707 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:59.826740 25710 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:59.829254 25709 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:59.759577 31476 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:19:59.800624 31391 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:59.830472 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.836324 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.856721 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.857496 25779 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:19:59.840837 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:19:59.817591 25629 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
W20260709 12:19:59.842370 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.822081 25712 pending_rounds.cc:85] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 11
I20260709 12:19:59.859573 25712 pending_rounds.cc:107] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:19:59.859859 25712 raft_consensus.cc:2889] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:59.845379 25704 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:19:59.834080 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:19:59.847885 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.808624 31346 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.803278 31239 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:19:59.758998 31179 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.788446 31337 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:19:59.802949 31381 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:59.799463 31444 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:19:59.788764 31483 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:19:59.862629 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:19:59.887459 25779 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:59.895704 25779 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:19:59.865017 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:19:59.812018 31368 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:19:59.862079 31346 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:59.870229 31476 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.814625 31401 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.813869 31292 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:59.876705 31488 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:19:59.878545 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 169773us
W20260709 12:19:59.879462 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45465.4us
I20260709 12:19:59.968266 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:59.879397 31192 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:19:59.854027 31485 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:19:59.910809 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:19:59.818357 31225 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:19:59.884380 31385 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:59.884109 31494 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.988659 31494 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:59.991156 31494 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:19:59.973116 25677 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:19:59.992897 25677 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:59.994035 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:19:59.995525 31301 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:59.996344 31301 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:19:59.997520 31301 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:19:59.976734 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2758) took 2122 ms. Trace:
I20260709 12:19:59.958093 31420 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:59.984543 25779 raft_consensus.cc:1240] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:20:00.007743 31420 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:00.008642 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2790) took 1503 ms. Trace:
I20260709 12:20:00.009028 31420 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.009335 25779 rpcz_store.cc:276] 0709 12:19:58.505352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.983895 (+1478543us) service_pool.cc:224] Handling call
0709 12:19:59.984299 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.008157 (+ 23858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.008551 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:59.998391 31315 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.998732 31472 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:20:00.012038 31177 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:00.012987 31493 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:20:00.000432 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.001161 31421 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:59.846292 31015 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.015086 25784 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 18: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:00.015635 25633 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:20:00.017042 31015 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.885048 31055 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 12:20:00.002770 31385 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:00.014132 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1024) took 1892 ms. Trace:
I20260709 12:20:00.018821 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:00.019766 26337 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 18: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:00.018927 25786 rpcz_store.cc:276] 0709 12:19:58.121687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.649226 (+527539us) service_pool.cc:224] Handling call
0709 12:19:58.663466 (+ 14240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:58.664144 (+   678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:58.664172 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:58.674983 (+ 10811us) log.cc:844] Serialized 26 byte log entry
0709 12:19:59.588992 (+914009us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b1800356200. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:19:59.740312 (+151320us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:59.740363 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.740452 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:00.012355 (+271903us) spinlock_profiling.cc:243] Waited 7.2 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:00.013878 (+  1523us) raft_consensus.cc:1685] finished
0709 12:20:00.013913 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.014069 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24732,"raft.queue_time_us":158,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":64041088,"thread_start_us":151027,"threads_started":1,"wal-append.queue_time_us":1196438}
I20260709 12:20:00.020512 31378 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:00.022220 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2795) took 1492 ms. Trace:
I20260709 12:20:00.022641 25786 rpcz_store.cc:276] 0709 12:19:58.529701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.021362 (+1491661us) service_pool.cc:224] Handling call
0709 12:20:00.021704 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:00.021897 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:00.021925 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:00.021967 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:00.021997 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.022048 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.022163 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:59.984215 31459 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:20:00.024819 31387 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:20:00.025029 25786 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:20:00.014187 25709 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:00.026381 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2796) took 1496 ms. Trace:
I20260709 12:20:00.026795 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:00.027074 25786 rpcz_store.cc:276] 0709 12:19:58.529935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.024124 (+1494189us) service_pool.cc:224] Handling call
0709 12:20:00.024597 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.026003 (+  1406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.026299 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:00.028066 31494 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:00.028465 31494 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.028833 31494 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:59.867659 31391 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.019984 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2791) took 1493 ms. Trace:
I20260709 12:20:00.031076 25785 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11. Ops: []
I20260709 12:20:00.031817 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2277) took 1308 ms. Trace:
I20260709 12:20:00.032320 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:00.032229 25785 rpcz_store.cc:276] 0709 12:19:58.723266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.030575 (+1307309us) service_pool.cc:224] Handling call
0709 12:20:00.030910 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:00.031601 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.031767 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:00.033903 31231 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.034467 31231 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.034987 31387 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:20:00.036635 31231 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.037192 31231 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.037637 31231 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.038468 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:00.038913 31368 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.039790 31481 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:00.040203 31368 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:00.040302 31481 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.042133 25781 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:20:00.042749 31387 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:00.043635 31387 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:00.043962 31239 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.044741 31387 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.026535 25787 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 12:20:00.051860 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2797) took 1521 ms. Trace:
I20260709 12:20:00.052311 25787 rpcz_store.cc:276] 0709 12:19:58.530219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.025891 (+1495672us) service_pool.cc:224] Handling call
0709 12:20:00.026259 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:00.051432 (+ 25173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.051783 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:59.968087 31401 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:00.047631 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 65269.3us
I20260709 12:20:00.031960 25784 raft_consensus.cc:1240] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 12:19:59.653743 25783 rpcz_store.cc:276] 0709 12:19:58.122133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.070617 (+948484us) service_pool.cc:224] Handling call
0709 12:19:59.071043 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:59.071292 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:59.097239 (+ 25947us) log.cc:844] Serialized 14 byte log entry
0709 12:19:59.501503 (+404264us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b180045e740. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a 00007f29b5999a2a
0709 12:19:59.613097 (+111594us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:59.613191 (+    94us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:59.613265 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.613373 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:59.613545 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19360,"prepare.queue_time_us":25767,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":19539,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":33137,"spinlock_wait_cycles":11889920,"thread_start_us":91741,"threads_started":1}
I20260709 12:20:00.006765 25778 rpcz_store.cc:276] 0709 12:19:57.854551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.036171 (+181620us) service_pool.cc:224] Handling call
0709 12:19:59.458587 (+1422416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:59.458788 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:59.458812 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:59.458853 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:59.458891 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.458941 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:59.976496 (+517555us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001252d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:19:59.976659 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3270394240}
I20260709 12:20:00.031312 25779 rpcz_store.cc:276] 0709 12:19:58.526876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.019205 (+1492329us) service_pool.cc:224] Handling call
0709 12:20:00.019488 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:00.019671 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:00.019698 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:00.019744 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:20:00.019775 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.019825 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.019929 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:00.018761 26307 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:00.059109 25783 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:20:00.050120 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.045643 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2788) took 1558 ms. Trace:
I20260709 12:20:00.059739 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2281) took 1332 ms. Trace:
I20260709 12:20:00.060141 25783 rpcz_store.cc:276] 0709 12:19:58.727387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.058409 (+1331022us) service_pool.cc:224] Handling call
0709 12:20:00.058772 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:00.059505 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.059688 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:00.061141 25709 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:00.063699 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2793) took 1536 ms. Trace:
I20260709 12:20:00.054169 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2280) took 1327 ms. Trace:
I20260709 12:19:59.886646 31196 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.163s
I20260709 12:20:00.064095 25784 rpcz_store.cc:276] 0709 12:19:58.527408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.022036 (+1494628us) service_pool.cc:224] Handling call
0709 12:20:00.023508 (+  1472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:00.063418 (+ 39910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.063645 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:00.064361 25787 rpcz_store.cc:276] 0709 12:19:58.726412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.053145 (+1326733us) service_pool.cc:224] Handling call
0709 12:20:00.053470 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:00.053706 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:00.053770 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:00.053829 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:20:00.053881 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.053954 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.054093 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:00.066596 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:00.059943 25781 rpcz_store.cc:276] 0709 12:19:58.487665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.481211 (+993546us) service_pool.cc:224] Handling call
0709 12:20:00.041903 (+560692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.042545 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.045466 (+  2921us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54001dd3d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:00.045587 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257485568}
I20260709 12:20:00.046268 31179 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.032238 31391 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:19:59.958227 31483 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.054847 31177 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:19:59.980466 31216 consensus_queue.cc:1059] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 12:20:00.104422 25784 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:20:00.106179 25784 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:19:59.986627 31449 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 12:20:00.118198 25784 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:00.030351 31318 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:00.131178 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.131814 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.119717 31177 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:00.145784 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 281743us
I20260709 12:20:00.113195 31391 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:20:00.178848 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.179557 25710 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:00.183234 25709 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:00.174114 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:00.177538 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.186934 25601 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.956s (SENT) [suppressed 43 similar messages]
I20260709 12:20:00.100454 31292 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.185629 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:00.194743 31179 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:00.197611 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.149384 31177 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.204952 25787 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:20:00.205495 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.212838 25705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.019s	sys 0.040s
W20260709 12:20:00.213235 25705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.020s	sys 0.040s
I20260709 12:20:00.203665 31177 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.205101 31275 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:20:00.221024 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 154524us
I20260709 12:20:00.193784 31254 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.215554 31192 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:00.237687 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.226598 31179 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.208128 31369 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:20:00.220026 31231 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:00.213668 31055 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:20:00.245500 25705 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:00.227684 31275 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:00.243373 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 224675us
I20260709 12:20:00.268378 25786 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:20:00.242564 31391 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:00.274777 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:00.248203 31369 consensus_queue.cc:1243] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:20:00.249963 31275 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:20:00.323434 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.270264 31055 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:20:00.324564 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49883.9us
I20260709 12:20:00.244426 31179 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.276911 31409 raft_consensus.cc:3055] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:00.278607 31391 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:00.330430 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 273279us
I20260709 12:20:00.332804 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:00.351687 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.276614 31341 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:20:00.318423 31483 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:00.358851 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 171943us
I20260709 12:20:00.247318 31192 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:00.363023 31499 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 12:20:00.369526 25601 connection.cc:256] RPC call timeout handler was delayed by -1.0790416790000563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:20:00.326002 31055 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:00.331351 31409 raft_consensus.cc:740] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:00.350103 31391 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:00.355726 31511 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:20:00.423079 25713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.025s	sys 0.003s
W20260709 12:20:00.423822 25713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.026s	sys 0.003s
W20260709 12:20:00.427659 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60380us
I20260709 12:20:00.382723 31337 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:00.383878 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.433015 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:00.403687 31459 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.459471 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2286) took 1245 ms. Trace:
I20260709 12:20:00.460094 25787 rpcz_store.cc:276] 0709 12:19:59.214121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.067068 (+852947us) service_pool.cc:224] Handling call
0709 12:20:00.067398 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.205435 (+138037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.459046 (+253611us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540009a454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:00.459370 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315787008}
I20260709 12:20:00.461843 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:00.437879 31485 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:00.453387 31471 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.428808 31318 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:00.427886 31506 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:20:00.484926 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 152218us
I20260709 12:20:00.426857 31494 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.467556 31254 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:00.502310 31513 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:20:00.503028 31500 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:20:00.504681 31192 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.496263 31471 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.457170 31459 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.455083 31337 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.496923 31509 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:20:00.487375 31292 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.524983 31509 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:00.439088 31409 consensus_queue.cc:260] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:00.529894 31378 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.005s
I20260709 12:20:00.528149 31471 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:00.529829 31487 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.658s	user 0.002s	sys 0.000s
I20260709 12:20:00.495659 31506 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.525701 31479 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:00.488972 31485 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.520714 31511 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:00.543668 31512 log.cc:927] Time spent T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:20:00.544294 31494 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:00.538154 31506 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:00.548900 31511 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.580752 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:00.587621 25536 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:00.590874 25536 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:00.599627 25536 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 5 to 7, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 7 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.602742 25536 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.605988 25536 catalog_manager.cc:5697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 3 to 8, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 8 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.612689 25536 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 9 to 13, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 13 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.616904 25536 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:00.615576 25532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 196) took 19184 ms (client timeout 15000 ms). Trace:
W20260709 12:20:00.370940 25601 connection.cc:256] RPC call timeout handler was delayed by -1.0154769519999718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23950000000000002s for a response
I20260709 12:20:00.623297 25536 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:00.627619 25536 catalog_manager.cc:5697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.632000 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:00.620646 25532 rpcz_store.cc:269] 0709 12:19:41.431477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.454553 (+ 23076us) service_pool.cc:224] Handling call
0709 12:20:00.615303 (+19160750us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:52.252330 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:52.253053 (+   723us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:52.253104 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:19:52.253123 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:52.284054 (+ 30931us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:52.284166 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:52.284203 (+    37us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:19:52.285623 (+  1420us) tablet.cc:701] Row locks acquired
0709 12:19:52.285680 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:19:52.285869 (+   189us) write_op.cc:270] Start()
0709 12:19:52.286030 (+   161us) write_op.cc:276] Timestamp: P: 1783599592285800 usec, L: 0
0709 12:19:52.286070 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:19:52.529242 (+243172us) log.cc:844] Serialized 6345 byte log entry
0709 12:19:53.228925 (+699683us) op_driver.cc:464] REPLICATION: finished
0709 12:19:53.285686 (+ 56761us) write_op.cc:301] APPLY: starting
0709 12:19:53.302094 (+ 16408us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:53.638529 (+336435us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:53.638689 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:53.713636 (+ 74947us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:53.714133 (+   497us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:19:53.714225 (+    92us) write_op.cc:312] APPLY: finished
0709 12:19:53.716828 (+  2603us) log.cc:844] Serialized 148 byte log entry
0709 12:19:53.717815 (+   987us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:53.718041 (+   226us) write_op.cc:454] Released schema lock
0709 12:19:53.718143 (+   102us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:53.747201 (+ 29058us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:53.747246 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3295890,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":14217,"apply.run_cpu_time_us":28319,"apply.run_wall_time_us":503650,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71103,"num_ops":16,"prepare.queue_time_us":329610,"prepare.run_cpu_time_us":30160,"prepare.run_wall_time_us":542797,"raft.queue_time_us":21630,"raft.run_cpu_time_us":6932,"raft.run_wall_time_us":15152,"replication_time_us":942698,"thread_start_us":345355,"threads_started":4,"wal-append.queue_time_us":243948,"wal-append.run_cpu_time_us":10830,"wal-append.run_wall_time_us":1880365}]]}
I20260709 12:20:00.636325 25536 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 5 to 9, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 9 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.639055 25536 catalog_manager.cc:5697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 6 to 13, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 13 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.642335 25536 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 7 to 10, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 10 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:00.644678 25536 catalog_manager.cc:5697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 3 to 10, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 10 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:00.646867 25536 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 7 to 11, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 11 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:20:00.648365 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.649433 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.649055 25536 catalog_manager.cc:5697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:00.650080 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.651553 25536 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:20:00.660902 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.661546 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.664022 25536 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 11 to 15, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 15 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:00.671996 25709 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:00.674320 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 953) took 1142 ms. Trace:
I20260709 12:20:00.674958 25706 rpcz_store.cc:276] 0709 12:19:59.531506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.531646 (+   140us) service_pool.cc:224] Handling call
0709 12:20:00.673733 (+1142087us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b54002238d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:00.674040 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112416128}
I20260709 12:20:00.498539 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
W20260709 12:20:00.705271 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 206685us
W20260709 12:20:00.705610 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.545507 31459 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:00.706681 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.708105 31535 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:00.709024 31535 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:00.709463 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:00.538812 31479 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:00.547426 31485 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.675041 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.684721 25707 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:00.684962 31179 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:00.692750 31494 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:00.760932 31494 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.761443 26308 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:00.763880 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:00.764256 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:00.765555 31421 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:00.766675 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2798) took 2236 ms. Trace:
I20260709 12:20:00.766420 31421 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.760957 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2789) took 2270 ms. Trace:
I20260709 12:20:00.767396 25782 rpcz_store.cc:276] 0709 12:19:58.490080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.484479 (+994399us) service_pool.cc:224] Handling call
0709 12:19:59.497896 (+ 13417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:59.687976 (+190080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:59.688010 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:59.688055 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:59.688086 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.760297 (+1072211us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54001248d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:00.760405 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.760708 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436599040}
I20260709 12:20:00.769513 31421 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:00.694909 31369 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:00.772471 31369 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.773372 31471 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:00.696244 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2283) took 1966 ms. Trace:
I20260709 12:20:00.774168 31471 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:00.775115 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.774097 25779 rpcz_store.cc:276] 0709 12:19:58.729183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.059335 (+1330152us) service_pool.cc:224] Handling call
0709 12:20:00.059667 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.068964 (+  9297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:00.069011 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:00.069548 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:20:00.298667 (+229119us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:00.387253 (+ 88586us) log.cc:844] Serialized 14 byte log entry
0709 12:20:00.428728 (+ 41475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.429142 (+   414us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:00.695648 (+266506us) raft_consensus.cc:1685] finished
0709 12:20:00.695690 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.696007 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41184,"prepare.queue_time_us":206889,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":52342,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":3653,"spinlock_wait_cycles":9440512,"thread_start_us":247987,"threads_started":2,"wal-append.queue_time_us":306873}
I20260709 12:20:00.776438 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:00.701725 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 268532us
I20260709 12:20:00.777698 25782 raft_consensus.cc:2368] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:00.778869 31498 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.779472 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.779704 31498 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.780537 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1100) took 1303 ms. Trace:
I20260709 12:20:00.781111 26335 rpcz_store.cc:276] 0709 12:19:59.476842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.778964 (+1302122us) service_pool.cc:224] Handling call
0709 12:20:00.780415 (+  1451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:00.782996 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:00.783262 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.717355 31341 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.784642 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:00.786077 26337 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.720577 31391 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.722045 31479 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.767113 25786 rpcz_store.cc:276] 0709 12:19:58.530560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.028998 (+1498438us) service_pool.cc:224] Handling call
0709 12:20:00.029620 (+   622us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:00.269018 (+239398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.766032 (+497014us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54002585d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:00.766447 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512773376}
I20260709 12:20:00.724498 31337 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.726792 31292 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:00.728296 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:00.729976 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:00.710667 25712 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.730326 31508 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:00.464071 31055 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.796556 25633 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:00.759805 25711 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:00.695006 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2282) took 1966 ms. Trace:
I20260709 12:20:00.713747 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.785800 31341 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.777221 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:00.787688 31015 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:00.784694 31027 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:00.777711 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.719878 31368 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.788858 31190 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:00.791728 25781 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:00.785069 31378 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.802976 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:00.787846 31084 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:00.814915 31084 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:00.815670 25636 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:20:00.815986 31391 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:00.816325 31084 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.818938 31368 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.805023 31471 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.820540 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.820688 31533 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:00.811273 31190 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:00.819164 31528 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:20:00.807507 31027 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:00.831496 31369 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.800325 25778 rpcz_store.cc:276] 0709 12:19:58.728391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.059211 (+1330820us) service_pool.cc:224] Handling call
0709 12:20:00.059516 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.059893 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:00.059922 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:00.070324 (+ 10402us) log.cc:844] Serialized 26 byte log entry
0709 12:20:00.480404 (+410080us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:00.480448 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.480518 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:00.694250 (+213732us) raft_consensus.cc:1685] finished
0709 12:20:00.694299 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.694710 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13313,"raft.queue_time_us":5033,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6710,"thread_start_us":367751,"threads_started":1,"wal-append.queue_time_us":415454}
I20260709 12:20:00.834936 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:00.836722 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:00.838171 31190 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.838943 31015 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.839721 31190 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:00.843451 26337 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:00.844108 31315 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:00.843796 31027 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.844669 31315 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:00.846155 31346 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.879s	user 0.009s	sys 0.015s
I20260709 12:20:00.847308 31378 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.846160 31315 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.850719 31315 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:00.851851 31341 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.852504 31341 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:00.853219 31292 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.856799 25707 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:00.858513 25637 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:00.858924 31507 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:00.822009 31368 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.859546 31507 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.859985 31507 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:00.830394 31498 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:00.838274 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2279) took 2031 ms. Trace:
I20260709 12:20:00.838196 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:00.872879 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:00.873565 31381 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.874447 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:00.874320 31381 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.877632 31055 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:00.877867 31534 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:20:00.880407 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:00.881060 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:00.870167 31508 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.848259 31419 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:00.888262 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.889120 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.892729 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:00.894587 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:00.894640 31491 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.023s	user 0.000s	sys 0.002s
I20260709 12:20:00.892966 25677 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:00.893170 31471 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:20:00.901142 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2862) took 1392 ms. Trace:
I20260709 12:20:00.883930 25779 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.902849 31481 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:00.903308 25632 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:00.904492 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:00.904528 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:00.885505 31479 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:00.902287 31471 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:00.892043 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:00.915069 31498 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:00.808171 31535 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:00.917258 31514 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.531s	user 0.000s	sys 0.002s
I20260709 12:20:00.919277 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:00.888082 31419 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:00.923477 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 131254us
I20260709 12:20:00.892339 31196 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:00.927024 31498 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:00.927531 31498 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:00.927950 31498 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:00.901549 25708 rpcz_store.cc:276] 0709 12:19:59.508878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.509123 (+   245us) service_pool.cc:224] Handling call
0709 12:19:59.528996 (+ 19873us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:59.529294 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:59.529332 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:59.529925 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:19:59.650890 (+120965us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:59.650931 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.651002 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:00.893485 (+1242483us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:00.900644 (+  7159us) raft_consensus.cc:1685] finished
0709 12:20:00.900670 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.900941 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17224064,"thread_start_us":101942,"threads_started":1,"wal-append.queue_time_us":286006}
W20260709 12:20:00.931406 31177 consensus_meta.cc:287] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.008s
I20260709 12:20:00.931877 31177 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.883680 31494 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:00.868836 25785 rpcz_store.cc:276] 0709 12:19:58.725689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.033694 (+1308005us) service_pool.cc:224] Handling call
0709 12:20:00.034112 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.218910 (+184798us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:20:00.348455 (+129545us) log.cc:844] Serialized 14 byte log entry
0709 12:20:00.474717 (+126262us) log.cc:844] Serialized 14 byte log entry
0709 12:20:00.475120 (+   403us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:00.698099 (+222979us) log.cc:844] Serialized 26 byte log entry
0709 12:20:00.698511 (+   412us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:00.699296 (+   785us) log.cc:844] Serialized 14 byte log entry
0709 12:20:00.699676 (+   380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.699849 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:00.711258 (+ 11409us) raft_consensus.cc:1685] finished
0709 12:20:00.711281 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.756875 (+ 45594us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":302802,"prepare.run_cpu_time_us":159,"prepare.run_wall_time_us":163,"raft.queue_time_us":12802,"raft.run_cpu_time_us":682,"raft.run_wall_time_us":12237,"thread_start_us":419234,"threads_started":3,"wal-append.queue_time_us":180998}
I20260709 12:20:00.935374 25784 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:00.936270 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:00.936612 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.911042 31471 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.942399 31471 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:00.924178 25782 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:00.942453 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.937548 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:00.937934 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1039) took 1073 ms. Trace:
I20260709 12:20:00.940712 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:00.946291 31510 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:00.943059 25708 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:00.946134 25710 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:00.948562 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.949317 26343 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:00.949893 25786 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:00.952149 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.952802 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.953102 25706 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:00.954178 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.954720 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.954898 25711 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:20:00.950351 25784 rpcz_store.cc:276] 0709 12:19:59.864367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.119104 (+254737us) service_pool.cc:224] Handling call
0709 12:20:00.121960 (+  2856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:00.935928 (+813968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.937651 (+  1723us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b540011f154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:00.937884 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866830080}
I20260709 12:20:00.956468 25708 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:20:00.957561 31275 consensus_meta.cc:287] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.008s
I20260709 12:20:00.952145 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.959439 25710 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:00.958303 25784 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:00.960577 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:00.960486 31496 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:00.958863 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.888124 31508 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:00.883945 31378 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:00.937003 25785 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:00.968569 25778 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:20:00.970291 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 110195us
I20260709 12:20:00.971276 25786 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:00.950343 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:00.847342 31346 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:00.974313 25712 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:00.958068 31275 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:00.947252 31391 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.982091 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.984764 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:00.994870 31541 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:00.995543 31541 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.996057 31541 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.996625 25637 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:00.995301 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:00.998905 26336 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:20:00.998930 31177 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:01.002719 25710 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:01.005589 31391 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.007721 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 948) took 1525 ms. Trace:
I20260709 12:20:01.008190 25713 rpcz_store.cc:276] 0709 12:19:59.481936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.482066 (+   130us) service_pool.cc:224] Handling call
0709 12:19:59.493951 (+ 11885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:59.494254 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:59.815360 (+321106us) log.cc:844] Serialized 14 byte log entry
0709 12:20:00.910581 (+1095221us) log.cc:844] Serialized 14 byte log entry
0709 12:20:00.986068 (+ 75487us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:00.988685 (+  2617us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:01.007122 (+ 18437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:01.007266 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:01.007494 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":837715,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":106,"raft.queue_time_us":6296,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":41613,"thread_start_us":1300550,"threads_started":3,"wal-append.queue_time_us":717298}
I20260709 12:20:01.011035 31027 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.011222 25784 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:01.011871 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.018425 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.004977 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:01.034984 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.035136 31391 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:01.035529 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.036360 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:01.037586 25629 raft_consensus.cc:2410] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:20:01.011271 31346 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:01.039050 31510 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.039454 31369 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:01.039992 31510 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.040239 31369 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:01.040884 31401 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.971s	user 0.010s	sys 0.009s
I20260709 12:20:01.041329 31401 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.042997 31510 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.043190 31369 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:01.052738 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.051864 25779 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:01.033934 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.050989 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:01.073128 31503 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.623s	user 0.000s	sys 0.000s
I20260709 12:20:01.074991 31292 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.076184 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:01.077632 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 163254us
I20260709 12:20:01.078116 25787 raft_consensus.cc:2368] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18.
I20260709 12:20:01.079749 31421 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:01.080621 31084 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:01.087344 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.088168 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.091150 31510 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:01.091691 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.091909 31292 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:20:01.092449 31506 consensus_meta.cc:287] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.012s
I20260709 12:20:01.092681 31292 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:01.092943 31506 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.096165 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.097993 26342 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:20:01.098609 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:01.099242 25713 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.101402 31254 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:01.101886 31254 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:01.102988 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 98282.3us
I20260709 12:20:01.103052 31254 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:01.107712 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.108160 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.108592 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.109647 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.110082 26338 raft_consensus.cc:1240] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:01.088204 26340 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:01.118533 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:01.119226 25712 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:01.118921 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:01.120913 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.126103 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.126186 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.131383 26335 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:01.126416 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:01.132448 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.091698 31510 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:01.138172 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 59718.5us
W20260709 12:20:01.140250 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 221180us
I20260709 12:20:01.140645 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.141590 26340 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:20:01.140275 26341 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:20:01.143903 31301 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:01.144526 31301 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:01.150278 26344 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:20:01.151298 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:01.153493 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:01.154574 31387 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:01.155448 31387 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:01.156067 31387 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:01.157272 31387 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.158123 25630 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:01.158833 31292 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.161311 31254 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:01.160764 31471 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.168243 25708 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:01.168955 25779 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:20:01.170707 31240 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.171296 31240 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.171911 31401 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:01.172114 31421 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:20:01.174069 31506 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.123611 31391 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:20:01.180671 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.181324 26343 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:01.181753 26343 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:01.182885 31528 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:01.183446 31528 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:01.184746 31528 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:01.185503 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 81493.8us
I20260709 12:20:01.186545 31507 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.189075 31507 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.190668 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.186774 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.145691 31301 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:01.193878 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.170323 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:01.182628 26343 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:01.202819 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 59493.7us
I20260709 12:20:01.180724 31275 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:01.208923 26343 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:01.213411 25782 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:01.214010 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2802) took 1042 ms. Trace:
I20260709 12:20:01.214425 25782 rpcz_store.cc:276] 0709 12:20:00.171540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.791794 (+620254us) service_pool.cc:224] Handling call
0709 12:20:01.213946 (+422152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:01.216634 25778 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:01.139220 31510 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:01.217445 25713 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:01.214891 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:01.216210 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:01.227206 25637 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:01.231076 25785 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:20:01.232537 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.191128 31177 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:01.231700 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.236423 26337 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.240928 25787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2809) took 1067 ms. Trace:
I20260709 12:20:01.241485 25787 rpcz_store.cc:276] 0709 12:20:00.173455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.461431 (+287976us) service_pool.cc:224] Handling call
0709 12:20:01.240609 (+779178us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54001a72d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:01.240832 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393796864}
I20260709 12:20:01.243703 31239 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:20:01.249380 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62797.4us
I20260709 12:20:01.205492 31015 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:01.260437 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:01.262208 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46105us
I20260709 12:20:01.229398 31471 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:01.265007 25706 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:01.261550 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:01.276998 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:01.276692 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44195us
I20260709 12:20:01.251219 31479 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:01.263859 31401 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:20:01.280035 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 62131.2us
W20260709 12:20:01.293882 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.294713 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:01.295876 25636 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:01.296423 25784 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:01.254457 31381 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.298002 25629 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:01.296486 25636 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:01.295953 31015 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:01.298761 31507 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:01.304347 26307 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:01.268270 31506 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:20:01.268576 31177 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.301517 31015 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:01.304087 31401 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:01.327500 25636 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.328068 31534 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:01.328387 26307 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:01.328537 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:01.328653 31476 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:01.328926 31192 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
W20260709 12:20:01.337174 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 86814.6us
I20260709 12:20:01.338598 31015 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:01.348085 25712 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:01.349694 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.350769 31084 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:20:01.338613 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 162986us
I20260709 12:20:01.351398 31084 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:01.352514 31084 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.354840 31381 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.355588 31381 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.357339 25706 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:01.344574 31528 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.358954 25603 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.360131 31190 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:01.360694 31190 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:01.319940 31535 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:20:01.361824 31190 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.373449 31225 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.374105 31225 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.374822 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.376585 31225 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.377969 25636 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:01.381676 31177 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.383985 31510 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.384886 31541 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.386214 25711 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:01.388348 31401 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.389588 26343 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:01.395435 31391 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:20:01.395519 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 67209.8us
I20260709 12:20:01.397913 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:01.399271 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:01.400833 31254 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:01.405809 26337 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:01.413195 25637 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:01.400828 31534 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:20:01.421298 31543 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:01.421682 31509 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:20:01.424078 31240 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:01.427145 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52523.9us
W20260709 12:20:01.428521 31519 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.946s	user 0.002s	sys 0.000s
I20260709 12:20:01.431258 31541 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:01.420868 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.424113 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.440281 31254 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:01.441280 31254 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:01.443449 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.445241 31476 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:01.453799 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.453076 25750 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:01.456593 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.436162 31543 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:01.460857 25778 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:20:01.384891 31498 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.470563 25783 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:20:01.488430 25783 pending_rounds.cc:85] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 8
I20260709 12:20:01.482479 31508 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:20:01.488699 25783 pending_rounds.cc:107] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:20:01.489012 25783 raft_consensus.cc:2889] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:01.482803 31505 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:01.489764 31055 consensus_meta.cc:287] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.611s	user 0.002s	sys 0.009s
I20260709 12:20:01.490254 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:01.490392 31505 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.492522 31498 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:01.494164 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2800) took 2609 ms. Trace:
I20260709 12:20:01.495472 31342 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:01.473495 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2784) took 3280 ms. Trace:
I20260709 12:20:01.462594 31543 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:01.467361 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2805) took 1295 ms. Trace:
I20260709 12:20:01.542621 25636 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:20:01.496902 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.543946 26344 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.480952 31225 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
W20260709 12:20:01.498011 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 54796.8us
I20260709 12:20:01.498588 31505 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:01.503237 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 144348us
I20260709 12:20:01.503931 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2424) took 1302 ms. Trace:
I20260709 12:20:01.504751 31564 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:01.506054 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:01.551164 25637 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.506134 25677 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.509687 25638 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:01.542011 25781 rpcz_store.cc:276] 0709 12:20:00.172384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.763442 (+591058us) service_pool.cc:224] Handling call
0709 12:20:01.467084 (+703642us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b54000870d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:01.467279 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61243136}
I20260709 12:20:01.513413 25601 leader_election.cc:400] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:01.516719 31476 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:01.519682 31381 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:01.478703 31419 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.551s	user 0.013s	sys 0.000s
I20260709 12:20:01.522797 31508 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:01.525372 25783 rpcz_store.cc:276] 0709 12:19:58.884943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.063913 (+1178970us) service_pool.cc:224] Handling call
0709 12:20:00.064172 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:01.489484 (+1425312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:01.493883 (+  4399us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54001ddb54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:01.494111 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200115968}
I20260709 12:20:01.456429 31510 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.453666 31534 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:01.510679 31497 consensus_queue.cc:1059] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:20:01.555583 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.555779 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49717.1us
I20260709 12:20:01.550733 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.496462 31239 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:01.555119 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.532183 25780 rpcz_store.cc:276] 0709 12:19:58.193335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.104381 (+911046us) service_pool.cc:224] Handling call
0709 12:19:59.441995 (+337614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:59.442321 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:59.442363 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:59.445388 (+  3025us) log.cc:844] Serialized 26 byte log entry
0709 12:19:59.591402 (+146014us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:59.591452 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.591536 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:00.529254 (+937718us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:00.696900 (+167646us) raft_consensus.cc:1685] finished
0709 12:20:00.696953 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:01.473066 (+776113us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400099f50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:01.473251 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806818944,"thread_start_us":116557,"threads_started":1,"wal-append.queue_time_us":350443}
I20260709 12:20:01.472463 31479 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:01.548067 25634 rpcz_store.cc:276] 0709 12:20:00.201028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.201197 (+   169us) service_pool.cc:224] Handling call
0709 12:20:00.201586 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.214539 (+ 12953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:00.214572 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:00.217125 (+  2553us) log.cc:844] Serialized 26 byte log entry
0709 12:20:00.326164 (+109039us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:00.326207 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.326282 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:01.416819 (+1090537us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:01.503386 (+ 86567us) raft_consensus.cc:1685] finished
0709 12:20:01.503407 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:01.503665 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28157,"raft.queue_time_us":36802,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5712,"spinlock_wait_cycles":5353728,"thread_start_us":108717,"threads_started":1,"wal-append.queue_time_us":205724}
I20260709 12:20:01.574328 31564 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:01.577791 31497 raft_consensus.cc:3055] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:01.578459 31497 raft_consensus.cc:740] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:01.579425 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 66282.9us
I20260709 12:20:01.579963 31497 consensus_queue.cc:260] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.580752 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.581487 31566 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:01.566059 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.541765 31240 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:01.570624 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.591714 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.582223 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.550591 31254 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:20:01.563282 31419 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.591130 26336 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:01.569478 31381 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:01.581970 31497 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.560467 31387 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:01.485632 31015 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.578545 31449 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:01.591423 31342 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:01.604532 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:01.610076 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.617404 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55369.5us
I20260709 12:20:01.608131 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:01.614190 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.621762 25781 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:01.614238 25778 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:20:01.612669 26343 raft_consensus.cc:2410] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:20:01.628865 25780 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:01.597918 31240 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.637200 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:01.637887 25779 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:01.638649 31225 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.639376 31225 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.642566 31225 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:01.643150 31254 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:01.643461 31409 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:01.647089 31015 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:01.648425 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.649454 26338 raft_consensus.cc:2410] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 19 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:20:01.648022 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.650581 26336 raft_consensus.cc:2368] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:01.650871 31534 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.656020 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.656479 31459 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.656996 31459 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.660030 31459 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:01.660133 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.660879 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.661685 26342 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:01.605532 31055 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:01.601485 31387 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:01.605804 31476 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:01.692981 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.692530 26335 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:01.594743 31508 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.613056 31449 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:01.701905 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:01.714303 25637 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:01.716486 25779 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:01.722890 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.720571 25630 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:01.612993 31381 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.733891 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.741878 31544 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:01.742029 31449 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.743085 31518 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:20:01.744088 31418 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 12:20:01.748854 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 134833us
I20260709 12:20:01.751143 31387 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.766518 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:01.768640 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.751192 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:01.784341 26344 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:01.784490 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.804342 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:01.811160 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60127.6us
I20260709 12:20:01.811431 26335 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:01.754410 31508 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.759555 31419 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:01.768043 31381 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:01.816835 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 120888us
I20260709 12:20:01.761826 31543 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.822475 26343 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:01.823753 31192 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:01.840865 31192 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:01.841969 31192 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.844278 26339 raft_consensus.cc:2410] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:20:01.844437 31543 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:01.845306 26341 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:20:01.862056 31240 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.864564 26336 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:01.877867 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:01.891464 31240 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:01.897508 31497 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:01.902050 31497 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:01.901935 31240 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.903232 31497 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.904486 31240 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.905645 25601 leader_election.cc:400] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18.
I20260709 12:20:01.905968 25601 leader_election.cc:403] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:20:01.906003 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.907157 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:01.907356 31507 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:01.907845 31507 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:01.908805 31507 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:01.909777 31528 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:01.910023 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.910780 31518 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.911000 25712 raft_consensus.cc:2410] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:20:01.911952 31543 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:01.912420 31543 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:01.912811 31543 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.913997 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.914651 25708 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:01.915040 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.911957 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.928808 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.929070 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:01.930267 25708 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:01.931041 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.934157 25707 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:01.946957 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:01.947795 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.947152 31507 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.948382 25706 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:01.933660 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.947773 25779 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:01.948401 26337 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:20:01.952544 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.957029 25786 raft_consensus.cc:2955] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 7 to 11, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:01.963351 31579 log.cc:927] Time spent T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:20:01.964679 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:01.967185 31498 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:20:01.967638 31498 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:01.968065 31498 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:01.932013 31533 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.970728 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:01.951115 31419 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:01.976557 25704 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:01.976915 31419 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:01.979363 31419 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:01.980108 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.981544 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:01.984241 25601 leader_election.cc:400] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:01.984611 25601 leader_election.cc:403] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:01.986820 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:01.999792 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:01.990136 31192 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:02.000581 26340 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:20:01.995317 31239 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:02.014721 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:02.012990 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:02.018662 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.019752 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:01.998889 31481 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.047781 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:01.995715 31533 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:02.058445 25676 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.773s (SENT)
I20260709 12:20:02.020457 31510 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:02.075230 31084 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.075989 31084 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.080276 31084 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:02.091486 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:02.109534 25704 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
W20260709 12:20:02.114775 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.045862 31239 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.090334 31518 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18.
W20260709 12:20:02.311758 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 327649us
I20260709 12:20:02.277537 31240 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:02.284139 31190 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:02.354985 31240 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.363727 31240 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:02.337100 31510 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.361876 31190 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:02.338414 31418 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:02.375237 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 327522us
W20260709 12:20:02.388625 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 342134us
W20260709 12:20:02.377630 31583 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.358s	user 0.001s	sys 0.001s
I20260709 12:20:02.390604 25630 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:02.369481 31541 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:02.404593 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 97581.3us
I20260709 12:20:02.304445 31481 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.413751 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.411880 26337 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:20:02.415104 26337 pending_rounds.cc:85] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 11
I20260709 12:20:02.415439 26337 pending_rounds.cc:107] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:20:02.415735 26337 raft_consensus.cc:2889] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:02.430979 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 62789.1us
W20260709 12:20:02.433869 25676 leader_election.cc:336] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.861s (SENT)
I20260709 12:20:02.434597 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:02.443842 25779 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:20:02.447211 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.435866 31381 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:02.459225 31381 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.463491 31533 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:02.421480 31510 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.431270 31418 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.467425 31418 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.469125 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1054) took 1061 ms. Trace:
I20260709 12:20:02.422040 31471 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.469578 25786 rpcz_store.cc:276] 0709 12:20:01.408046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.914647 (+506601us) service_pool.cc:224] Handling call
0709 12:20:01.915031 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:01.926351 (+ 11320us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:01.998105 (+ 71754us) log.cc:844] Serialized 14 byte log entry
0709 12:20:02.367403 (+369298us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:02.375599 (+  8196us) log.cc:844] Serialized 26 byte log entry
0709 12:20:02.375856 (+   257us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:02.375920 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:02.376032 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:02.468879 (+ 92847us) raft_consensus.cc:1685] finished
0709 12:20:02.468908 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:02.469055 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10541,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5625,"thread_start_us":368876,"threads_started":1,"wal-append.queue_time_us":432425}
I20260709 12:20:02.470798 31471 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.473527 31471 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:02.419703 31541 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.476481 31541 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:02.477160 31541 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:02.478410 25601 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:02.477573 31481 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:02.481108 25601 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:02.483016 31502 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:02.484375 31084 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:02.486387 31507 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:20:02.489944 26339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1102) took 1713 ms (client timeout 2190 ms). Trace:
W20260709 12:20:02.490562 26339 rpcz_store.cc:269] 0709 12:20:00.776309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.782379 (+  6070us) service_pool.cc:224] Handling call
0709 12:20:02.489619 (+1707240us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54004d6554. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:02.489830 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2436854016}
I20260709 12:20:02.492743 31239 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:02.499401 31568 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.004s	user 0.006s	sys 0.008s
I20260709 12:20:02.512888 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:02.520423 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.520648 25636 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:02.531554 31481 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:02.532212 31481 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.537897 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:02.470470 31418 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:02.539386 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:02.526901 31498 consensus_meta.cc:287] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.000s
I20260709 12:20:02.542675 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.546761 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.526576 31015 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:20:02.557791 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 131674us
W20260709 12:20:02.558907 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46255.8us
I20260709 12:20:02.539227 31510 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:02.567514 25704 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:20:02.540386 31391 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:02.559015 25751 leader_election.cc:336] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.511s (SENT)
I20260709 12:20:02.524549 31190 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.574815 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.572518 31595 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.580997 25636 raft_consensus.cc:1275] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:02.566169 31418 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:20:02.584069 25676 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.882s (SENT)
I20260709 12:20:02.586231 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:02.567971 31533 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.548745 31498 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.596305 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.597096 25636 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:02.578414 31190 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:02.603003 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:02.604868 25676 leader_election.cc:336] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.915s (SENT)
I20260709 12:20:02.575666 31015 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:20:02.595005 31595 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.607944 31498 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:02.609254 31595 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:02.613601 25634 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 9. Current term is 10. Ops: []
I20260709 12:20:02.589609 31418 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:02.596771 31543 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.683s	user 0.014s	sys 0.000s
W20260709 12:20:02.635128 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 57444.2us
I20260709 12:20:02.596539 31533 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.643280 31481 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:02.575291 31391 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.609284 31381 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:02.648269 25676 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Timed out: RequestConsensusVote RPC to 127.23.217.68:35023 timed out after 1.963s (SENT)
I20260709 12:20:02.650779 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.651144 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:02.645859 31418 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:02.655916 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.651466 25630 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:02.661967 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:02.662163 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:02.666173 26339 raft_consensus.cc:2410] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:20:02.647774 31543 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.671324 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.672109 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.675010 31378 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.675583 31378 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.677876 31378 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:02.678045 31574 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.926s	user 0.029s	sys 0.019s
I20260709 12:20:02.679500 31543 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:02.673156 25708 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 11.
I20260709 12:20:02.681730 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 992) took 1024 ms. Trace:
I20260709 12:20:02.682816 31544 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:02.683346 31544 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:02.667085 31459 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:02.690347 31391 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:02.692435 31584 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.343s	user 0.001s	sys 0.001s
I20260709 12:20:02.703498 31533 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:02.718998 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45911.5us
I20260709 12:20:02.721817 25784 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:02.722754 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:02.723713 25631 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:02.724997 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.726018 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.726337 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.726821 25635 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:02.727027 25632 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:02.730295 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:02.730897 31385 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:02.733987 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:02.737612 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.738564 25629 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:02.740022 25632 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:02.746969 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:02.682313 26336 rpcz_store.cc:276] 0709 12:20:01.657863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.658031 (+   168us) service_pool.cc:224] Handling call
0709 12:20:01.658332 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:01.867450 (+209118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:02.681339 (+813889us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540065c2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:02.681645 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473753856}
I20260709 12:20:02.766227 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.758200 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:02.759235 25636 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
W20260709 12:20:02.772949 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 154652us
I20260709 12:20:02.774117 25631 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:02.777351 25780 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:02.763707 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:02.786296 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:02.792274 25635 raft_consensus.cc:2410] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:20:02.756091 31421 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:02.767382 31502 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:02.818840 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:02.758842 31541 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:20:02.826901 25785 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:02.827286 25631 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:20:02.828987 25631 pending_rounds.cc:85] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 9
I20260709 12:20:02.829315 25631 pending_rounds.cc:107] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:20:02.829718 25631 raft_consensus.cc:2889] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:02.830235 25631 pending_rounds.cc:107] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:20:02.830547 25631 raft_consensus.cc:2889] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:02.771549 31564 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:20:02.832734 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.829367 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:02.849315 25635 raft_consensus.cc:2393] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:02.820533 31421 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:02.795289 31418 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:02.827980 31381 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:02.867300 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:02.873096 31595 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:20:02.875603 31601 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:02.877133 31418 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:02.878345 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:02.879269 25637 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:02.880066 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.883836 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.884620 25638 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:02.886471 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.887318 25634 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:20:02.890137 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.890556 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.891117 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.891341 25634 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:02.896152 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.899180 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.901801 31595 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:02.902371 31595 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:02.902865 31595 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:20:02.908741 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.914392 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.915340 25630 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
W20260709 12:20:02.916628 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.921267 25677 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:02.923861 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.818742 31459 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 12:20:02.929131 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:02.930406 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:02.936369 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.942612 25630 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:02.941948 25634 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:02.924711 31541 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:02.936059 31587 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:20:02.953118 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.953737 25637 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:02.852892 31240 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.957684 25638 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:02.941929 31592 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:02.964802 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:02.966326 25632 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:02.954314 31541 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.967643 31541 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:02.973016 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:02.975244 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:02.976163 25634 raft_consensus.cc:2393] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:02.976656 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:02.976979 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.977922 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.979130 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.977870 25634 raft_consensus.cc:2410] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:20:02.980268 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.980804 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.981388 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.982738 25634 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:02.985731 31564 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:20:02.989910 31573 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.232s	user 0.005s	sys 0.010s
I20260709 12:20:02.952014 31254 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:02.991248 31254 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.993783 31459 consensus_queue.cc:1059] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:02.994889 31459 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:02.995062 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:02.996241 25636 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:02.997032 31459 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:20:02.997452 31459 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:02.998868 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1282) took 2031 ms. Trace:
I20260709 12:20:02.999714 31566 raft_consensus.cc:3055] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:03.000134 31566 raft_consensus.cc:740] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:02.934609 31418 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:20:03.001065 31566 consensus_queue.cc:260] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.952236 31409 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:03.003547 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.003301 31409 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.004586 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:03.006577 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 153891us
I20260709 12:20:03.007314 31381 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:03.008630 31381 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:02.998924 31385 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:03.001489 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.030438 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.033073 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.045360 25637 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 10.
I20260709 12:20:03.027598 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:02.966647 31592 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.999436 25633 rpcz_store.cc:276] 0709 12:20:00.968002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.968128 (+   126us) service_pool.cc:224] Handling call
0709 12:20:00.976247 (+  8119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:00.987485 (+ 11238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:00.987509 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:00.998087 (+ 10578us) log.cc:844] Serialized 26 byte log entry
0709 12:20:01.523440 (+525353us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b18002cefc0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:20:01.690210 (+166770us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:01.690280 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:01.690416 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:02.988908 (+1298492us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:02.998634 (+  9726us) raft_consensus.cc:1685] finished
0709 12:20:02.998661 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:02.998805 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4913,"raft.queue_time_us":2345,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":543,"spinlock_wait_cycles":69537024,"thread_start_us":152228,"threads_started":1,"wal-append.queue_time_us":759129}
I20260709 12:20:03.063156 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.064432 25637 raft_consensus.cc:2410] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:20:03.065953 31225 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:03.087531 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.018153 31589 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:03.117650 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.119468 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.120682 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.046787 31534 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.044422 31385 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:03.006999 31419 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 12:20:03.121243 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 180842us
I20260709 12:20:03.053526 31479 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:03.122057 25637 raft_consensus.cc:2376] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 16. Re-sending same reply.
I20260709 12:20:03.029860 31239 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 12:20:03.136358 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:03.137115 25635 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:03.137367 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.138582 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.139403 25637 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:03.119218 31391 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.058629 31592 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:03.157358 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:03.157804 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.113448 31409 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.159745 25636 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:03.159799 26339 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:03.117992 31566 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:03.123579 31240 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:02.998445 31459 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.017431 31421 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.178392 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:03.165206 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 192285us
I20260709 12:20:03.164296 25634 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:20:03.178874 31505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.008s	sys 0.007s
I20260709 12:20:03.202039 31385 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.210279 31385 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:03.211284 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.217506 31387 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:03.174566 31589 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.165467 31418 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:03.158403 31479 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.227401 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:03.230996 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:03.158608 31534 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:03.236657 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 224177us
I20260709 12:20:03.231127 26339 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:03.238742 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:03.166931 31391 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.208585 31254 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:03.252115 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 71604.8us
W20260709 12:20:03.235936 31505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.008s	sys 0.007s
I20260709 12:20:03.257632 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:03.260350 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.258700 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:03.261116 25786 raft_consensus.cc:2368] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:03.247771 31418 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.268033 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:03.269140 31239 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:03.269767 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.241117 31225 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.272746 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:03.273780 26339 raft_consensus.cc:2368] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:03.275489 25633 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:20:03.256826 31588 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:03.280073 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:03.276468 31239 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:03.272558 31604 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.287483 25630 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:03.272614 31497 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:03.292531 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:03.298420 26335 raft_consensus.cc:2393] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:03.298149 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.302577 25786 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:03.306933 25787 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:20:03.306998 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.307183 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:03.311424 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:03.312558 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:03.313661 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:03.317000 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:03.317363 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.281783 31533 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:20:03.308434 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.317000 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.323951 25781 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:03.324488 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:03.325265 25779 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:03.314265 26340 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:03.291415 31507 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:03.285362 31239 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:03.320267 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:03.321332 26342 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:20:03.321849 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 68868us
I20260709 12:20:03.287242 31409 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:20:03.279189 31620 consensus_queue.cc:1048] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:20:03.281016 31595 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.329803 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.278565 31225 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.328389 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:03.330931 25780 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:20:03.336647 31617 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:03.337687 31594 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:20:03.283787 31254 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:20:03.348367 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:03.318198 31621 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:03.329643 31507 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.348402 26335 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:03.304486 31604 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.334343 31239 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:03.347930 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:03.381027 25781 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:03.379407 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:03.382174 25708 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:03.373888 25784 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
W20260709 12:20:03.370105 31544 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.686s	user 0.008s	sys 0.006s
I20260709 12:20:03.384387 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:03.379880 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:03.389782 25635 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:03.390133 25783 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:03.385097 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:03.381063 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 94134.6us
I20260709 12:20:03.367604 31594 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.331934 31497 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.392179 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:03.394696 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.378635 31564 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:03.378259 31507 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:20:03.399633 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 66238.7us
I20260709 12:20:03.347800 31541 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.403072 25781 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:20:03.406296 26335 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
W20260709 12:20:03.408742 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 122649us
I20260709 12:20:03.413610 31015 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.416371 25782 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:20:03.395445 31594 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:03.417627 25782 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:03.418627 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:03.419687 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:03.420202 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.432843 26335 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:20:03.435238 25780 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 12:20:03.444300 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.444316 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:03.414237 31015 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.394847 31544 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:03.453750 31627 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 12:20:03.455397 31615 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:20:03.446787 25781 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:03.445214 31534 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.445092 31418 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:03.490958 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 260025us
I20260709 12:20:03.506700 31599 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:03.507342 31635 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:20:03.527290 31628 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:20:03.545323 31479 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.498600 31588 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:03.555809 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 62803.1us
I20260709 12:20:03.565083 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:03.549916 31595 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:03.572767 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 168440us
I20260709 12:20:03.576128 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.580921 25711 raft_consensus.cc:2410] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:20:03.600948 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:03.601400 25708 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:03.594458 31632 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:03.551232 31479 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
W20260709 12:20:03.607334 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 184136us
I20260709 12:20:03.615610 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.617595 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.620460 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:03.621045 25637 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:03.628072 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 236042us
I20260709 12:20:03.605100 31449 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:03.646867 31497 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.572453 31385 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:20:03.659718 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:03.636596 31541 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.624048 31595 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:03.672325 25783 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:03.652195 31604 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:03.678272 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.659262 31533 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:03.681319 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 198368us
W20260709 12:20:03.679340 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64751.2us
W20260709 12:20:03.681213 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51808.2us
W20260709 12:20:03.663528 31641 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:20:03.507545 31599 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.710062 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:03.710454 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.713475 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:03.714123 25710 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:03.718242 25637 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:03.720446 25713 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:20:03.677246 31449 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:03.747924 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.727129 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:03.746047 25713 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
W20260709 12:20:03.752576 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60465.5us
I20260709 12:20:03.755559 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.759006 31449 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.761206 25704 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:20:03.761698 25704 pending_rounds.cc:85] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 10
I20260709 12:20:03.761927 25704 pending_rounds.cc:107] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:20:03.762156 25704 raft_consensus.cc:2889] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:03.763574 25704 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:20:03.711302 31589 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:03.719663 31544 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:03.766865 31645 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:03.767864 31652 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.768433 31652 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.769387 25601 leader_election.cc:400] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:03.769737 25601 leader_election.cc:403] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:03.685518 31595 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.770434 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:03.771576 25712 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:03.771968 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:03.772364 31595 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.713258 31015 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:03.759653 25706 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:03.755838 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:03.731204 31592 consensus_meta.cc:287] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 12:20:03.739907 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:03.779357 25709 raft_consensus.cc:2410] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:20:03.736521 31459 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:03.731913 31391 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:03.785071 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45266.3us
I20260709 12:20:03.732844 31599 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:03.770267 31507 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:03.802906 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.807248 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:03.804780 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:03.783520 31387 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.004s
I20260709 12:20:03.801656 31566 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:03.726413 31239 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.789656 31645 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:03.784232 31544 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:20:03.719373 31533 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:03.820502 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.826417 25711 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:20:03.829437 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.676594 31588 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.833472 25712 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:03.836688 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.840451 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.841194 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.844345 25710 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:03.847167 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.819126 31638 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.850307 25705 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:20:03.851827 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:03.854249 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:03.830932 31630 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.832068 31645 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:03.790091 31592 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.855650 25709 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
W20260709 12:20:03.860927 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 116615us
W20260709 12:20:03.863142 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 91223.8us
I20260709 12:20:03.866634 31508 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:03.849462 31566 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:03.870114 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:03.902658 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:03.903506 26337 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
W20260709 12:20:03.905229 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59959us
I20260709 12:20:03.865425 31645 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:03.920810 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.851238 31650 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:20:03.925037 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:03.932662 31533 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.938760 31239 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:03.939188 31507 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:03.927064 31541 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:03.959538 31541 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.966840 31566 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:03.867681 31630 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.923151 31497 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:03.972244 31640 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.972992 31640 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.925498 31648 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.978282 31648 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.981067 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:03.987185 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 117100us
I20260709 12:20:03.989017 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:03.962260 31541 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:03.996855 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 71906.5us
I20260709 12:20:04.000033 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:03.858109 31638 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.001897 31651 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.002566 31640 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:03.994319 31497 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:20:03.991722 31594 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.003s
I20260709 12:20:04.013885 31630 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:04.014786 31654 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:20:03.998315 31620 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:04.015534 31659 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:20:04.022814 31651 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.016919 31385 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.017557 31652 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.991724 31508 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:20:04.038820 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.020767 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.039707 25637 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:03.994009 31619 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.041639 31619 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.993592 31055 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:20:04.022806 31015 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:20:04.028095 31656 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:20:04.046676 31225 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:04.048600 26337 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:04.049930 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:04.032658 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:04.051316 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.051785 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.052691 31622 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:04.053051 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.053674 25637 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.054276 25637 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:20:04.055575 31592 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:04.034111 31381 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:04.035162 31651 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:04.035600 31385 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.058887 31638 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:04.060377 31632 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:03.990159 31239 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.060865 31632 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:04.061134 31239 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.030169 31648 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:04.022609 31599 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:04.063410 31604 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:20:04.064591 31620 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:04.065109 31604 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:04.060534 31385 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:04.066967 31620 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:03.996163 31645 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:04.067950 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.067962 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.080385 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.080470 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.068977 25707 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:20:04.081419 26343 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:20:04.071372 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.070691 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:04.085816 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.086169 25637 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:04.086939 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:04.054276 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.063664 31239 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:04.088156 31497 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:04.027038 31594 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:04.090683 31594 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:04.053429 31622 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:04.074893 31619 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:04.085031 31240 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:20:04.075188 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:04.093829 25709 raft_consensus.cc:2410] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:20:04.075809 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:04.076503 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:04.077167 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.069784 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:04.095103 25708 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 9.
I20260709 12:20:04.096606 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:04.097920 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.098817 25783 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
W20260709 12:20:04.103518 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49321.5us
I20260709 12:20:04.104542 25779 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:04.104888 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.105760 26343 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:04.109680 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.083949 31604 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.111608 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:04.112627 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:04.126369 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:04.127090 25713 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:04.113255 31381 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.084898 31055 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:20:04.115084 31409 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:04.117681 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:04.120321 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.121414 25787 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:04.121907 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:04.173177 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 51370.1us
I20260709 12:20:04.092993 31660 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:04.129386 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.108990 31239 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.208324 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:04.194280 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.125348 31594 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:04.215027 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 105923us
I20260709 12:20:04.215471 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.216216 25784 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:04.155740 31645 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:04.159346 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69270us
I20260709 12:20:04.166476 25637 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:04.174968 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.220881 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.348737 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:04.223084 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 93816.7us
I20260709 12:20:04.348812 25779 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:04.350262 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:04.304494 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.352238 25782 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:04.214906 31599 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:04.204058 31544 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:04.204836 31660 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:04.346837 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.349406 25637 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:04.244174 31594 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:04.353951 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 144068us
W20260709 12:20:04.346813 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 109317us
I20260709 12:20:04.349354 25778 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
W20260709 12:20:04.358981 31666 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.123s	user 0.000s	sys 0.014s
I20260709 12:20:04.359313 31535 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:04.371758 25677 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:04.388756 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.391271 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.370476 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.373984 25601 leader_election.cc:400] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16.
I20260709 12:20:04.401332 25707 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:04.126914 25705 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:04.367857 31510 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:20:04.372179 31509 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:04.365098 31638 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:20:04.407752 31640 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:04.408578 31592 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:04.409114 31648 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:04.409870 31508 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.410619 31225 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:04.415534 25752 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.437296 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:04.414157 31632 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:03.630817 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:04.438736 31381 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:04.439646 31381 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:20:04.440192 31381 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:04.440554 31632 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:04.440642 31381 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:20:04.441982 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 68280.6us
I20260709 12:20:04.443396 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:04.444563 31544 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:04.444796 31630 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:04.444960 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:04.445478 31630 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.445935 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:04.371138 31660 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:04.416443 31592 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:04.449118 31592 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:04.449935 31619 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:04.418771 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.438529 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.453313 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.472254 25632 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:04.472656 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.454221 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.456740 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.475170 25634 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:20:04.457319 25630 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:04.475898 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:04.157226 31622 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:04.403587 26309 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:04.475008 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.458804 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:04.358321 25713 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
W20260709 12:20:04.482951 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79479.4us
I20260709 12:20:04.483078 25636 raft_consensus.cc:3060] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:04.414229 31599 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:04.495410 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.496150 25631 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:04.496100 31599 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.498972 31254 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:04.499409 31254 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:04.499558 31665 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:04.499951 31665 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.500262 31254 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.501505 31254 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:04.484781 31225 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:04.506579 31240 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.417071 31636 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:04.507777 31636 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:04.508283 31619 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:04.508999 31636 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.510799 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.512595 31652 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:04.512828 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:04.512698 31479 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:04.415776 31239 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:20:04.514814 31239 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.484222 31055 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:04.516007 31055 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:04.516959 31055 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.500289 31665 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:04.419013 31508 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:20:04.518891 31508 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:04.519322 31508 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:04.496814 25638 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:04.506642 31632 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:04.485482 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66790.1us
I20260709 12:20:04.489003 25634 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:04.523782 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.525236 31240 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:04.486310 26307 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.525959 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.171818 31409 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:04.486919 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.531857 25637 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:04.537485 25636 raft_consensus.cc:2468] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
W20260709 12:20:04.539481 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 85357.8us
W20260709 12:20:04.535588 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 47544.5us
W20260709 12:20:04.544438 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60674.8us
I20260709 12:20:04.545876 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:04.543463 31543 raft_consensus.cc:2764] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:20:04.555305 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69069.6us
I20260709 12:20:04.556294 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:04.555349 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.559186 25634 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:04.543661 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:04.564934 25778 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:04.565438 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.566589 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.539258 31636 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:04.577275 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:04.581471 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.579979 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:04.569461 31507 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:04.561657 31239 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:04.565062 31544 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.585232 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:04.589035 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.551764 31479 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:04.593338 31479 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:04.593767 31479 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:04.594798 31479 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.595469 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:04.596323 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:04.597141 26342 raft_consensus.cc:2368] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:20:04.598578 31459 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:04.598852 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.599997 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:04.600579 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.600701 26335 raft_consensus.cc:2376] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16. Re-sending same reply.
I20260709 12:20:04.601713 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.598102 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.600951 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:04.600579 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.603493 26342 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:04.602022 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.572325 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:04.587899 31497 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:04.616475 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 47667.2us
I20260709 12:20:04.617671 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:04.622148 25636 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:04.587121 31507 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.572486 31535 consensus_queue.cc:1059] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:20:04.590142 31632 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:04.607251 31225 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:04.587687 31543 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.616341 31534 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:04.625725 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:04.607281 31566 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:20:04.602756 31479 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:04.610076 31648 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.632725 31507 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.617702 31622 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.628654 31664 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.607127 31595 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:20:04.631928 31632 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.627315 25636 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:04.637665 31225 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:04.619787 31660 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:04.631827 31652 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:20:04.641708 31672 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:20:04.663712 31544 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:04.665087 31660 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:04.679108 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:04.665616 25632 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:04.679759 25783 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:20:04.665475 31381 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:04.643996 31027 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:20:04.646279 31534 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.667565 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.655848 31632 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.669757 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.670938 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.649561 31566 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:04.668569 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.645483 31543 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.663028 31225 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:04.702075 25635 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:04.701556 25781 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:04.698447 31534 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:04.699324 31672 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:04.711313 31381 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:04.727190 25783 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:04.739766 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.754958 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.795460 25783 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:04.709921 31497 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:04.873073 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 273267us
I20260709 12:20:04.874106 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:04.887036 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.888052 25783 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
W20260709 12:20:04.894002 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.902834 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:04.910059 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 257503us
I20260709 12:20:04.911393 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:04.921113 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:04.897758 31648 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:04.890821 31588 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:04.928314 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.938400 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.940037 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.941388 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:04.942045 25783 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:20:04.944214 25782 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 10: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:04.935931 31648 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:04.979929 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 340365us
I20260709 12:20:04.884363 31566 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:04.992419 31679 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 12:20:04.903791 31509 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:05.009239 25783 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:05.011612 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:05.013176 31667 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:20:05.020260 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 127057us
W20260709 12:20:05.024050 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 74596.2us
I20260709 12:20:05.002637 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:04.995149 31645 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:05.034790 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:05.038697 25783 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:05.003861 31595 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.020586 31648 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:04.947749 31588 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:04.919409 31672 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.048605 25630 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:20:05.049500 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:05.055750 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:05.059438 25783 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:05.059631 25778 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:20:05.062065 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:05.023810 31665 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.037247 31543 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:05.065864 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:05.072980 26341 raft_consensus.cc:2368] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20.
I20260709 12:20:05.023736 31541 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 12:20:05.067574 31686 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:20:05.069823 25751 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.547s (SENT)
I20260709 12:20:05.079874 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:05.089583 25751 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.634s (SENT)
W20260709 12:20:05.094208 31682 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 12:20:05.095692 31684 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260709 12:20:05.096701 31683 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:20:05.097268 31685 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 12:20:05.098251 31225 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:05.100849 31225 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:05.101898 25785 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:05.110402 31622 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:05.067600 31381 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:05.087731 31254 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.002s
I20260709 12:20:05.135906 25629 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:05.033975 31509 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.051371 31595 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.133142 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:05.159597 26338 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:20:05.164511 26342 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.008s	sys 0.000s
W20260709 12:20:05.165025 26342 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.008s	sys 0.000s
W20260709 12:20:05.135812 31681 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:20:05.168270 26308 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:05.187793 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:05.199822 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 136574us
I20260709 12:20:05.175935 31535 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:05.216485 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.220618 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.221191 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.221619 25636 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:20:05.223387 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.196297 31254 raft_consensus.cc:2804] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:05.196519 31651 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:05.224860 25636 raft_consensus.cc:2368] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:20:05.227310 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.227597 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:05.205965 31507 consensus_meta.cc:287] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.003s
I20260709 12:20:05.224138 31693 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:05.231101 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.231376 31693 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.232440 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.233696 31693 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:05.237923 31508 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.002s
I20260709 12:20:05.230389 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.233711 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:05.261409 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.214323 31604 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.267346 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.213794 31535 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:05.274050 25634 raft_consensus.cc:2376] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 18. Re-sending same reply.
I20260709 12:20:05.276292 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.277865 25636 raft_consensus.cc:2393] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:05.279304 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.281175 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.288503 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:05.288527 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.290416 25638 raft_consensus.cc:2410] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 12 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:20:05.290874 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:05.291389 31640 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:05.291728 31664 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:05.292100 31664 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.292474 31664 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.292109 25751 leader_election.cc:400] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:20:05.295755 25751 leader_election.cc:403] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:05.296411 31640 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:20:05.296943 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:05.297896 31640 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:05.298595 31651 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:05.299892 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:05.299642 31651 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.300750 31640 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:05.301198 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:05.302089 31640 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:05.302554 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:05.303783 31622 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:05.304776 31664 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.305348 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:05.276674 31604 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:05.308122 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.309320 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.310046 31604 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:05.275741 31632 consensus_meta.cc:287] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.005s
I20260709 12:20:05.311422 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.311765 25634 raft_consensus.cc:2368] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
I20260709 12:20:05.315315 25636 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:05.311302 25631 raft_consensus.cc:2376] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 16. Re-sending same reply.
I20260709 12:20:05.318589 25633 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:05.273588 31693 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.328469 31409 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.329532 31632 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.238420 31508 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:05.331990 31508 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:05.332780 25780 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:20:05.277684 31535 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.263952 31507 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.277220 31700 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:20:05.328878 31693 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:05.325136 25707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50646 (request call id 984) took 1228 ms (client timeout 1619 ms). Trace:
I20260709 12:20:05.330108 31409 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:05.352211 25707 rpcz_store.cc:269] 0709 12:20:04.097072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.097195 (+   123us) service_pool.cc:224] Handling call
0709 12:20:05.324924 (+1227729us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54002251d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:05.325065 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527159168}
I20260709 12:20:05.322993 31588 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:05.348909 25710 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:05.350244 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:05.282761 31699 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:05.330343 31534 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.608s	user 0.002s	sys 0.005s
I20260709 12:20:05.372605 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:05.382032 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:05.317461 31622 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:05.386168 26339 raft_consensus.cc:2368] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19.
I20260709 12:20:05.384451 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:05.375180 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:05.378959 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:05.384132 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78868.5us
I20260709 12:20:05.398751 31695 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:05.387655 26338 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:05.400171 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.401252 25636 raft_consensus.cc:2368] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
W20260709 12:20:05.401737 25751 leader_election.cc:336] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.966s (SENT)
I20260709 12:20:05.409627 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:05.391073 31479 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.740s	user 0.013s	sys 0.006s
I20260709 12:20:05.418319 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:05.431607 26343 raft_consensus.cc:2368] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17.
I20260709 12:20:05.416531 31254 raft_consensus.cc:697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:05.399916 31699 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.450745 31254 consensus_queue.cc:237] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.388424 31534 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.462999 25711 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:05.467552 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.468521 25636 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 19: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:05.470816 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1072) took 1352 ms. Trace:
I20260709 12:20:05.471177 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.471187 25779 rpcz_store.cc:276] 0709 12:20:04.118746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.126413 (+  7667us) service_pool.cc:224] Handling call
0709 12:20:04.126692 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.350169 (+223477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.470590 (+1120421us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400086bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:05.470760 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318783744}
I20260709 12:20:05.457010 31381 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:05.469401 31509 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:05.469594 31664 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:05.483867 31715 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:20:05.410779 31695 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.470412 31595 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:05.471117 31588 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:20:05.464828 31700 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:05.549809 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 114477us
W20260709 12:20:05.642514 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 215983us
W20260709 12:20:05.633478 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 120566us
I20260709 12:20:05.644584 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:05.646409 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 236877us
I20260709 12:20:05.651325 25706 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:20:05.658874 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:05.668061 25707 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:05.675736 25706 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:05.675508 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:05.656028 31719 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.198s	user 0.003s	sys 0.007s
I20260709 12:20:05.667218 25751 leader_election.cc:400] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
I20260709 12:20:05.735375 25751 leader_election.cc:403] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:20:05.736526 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 69477.8us
I20260709 12:20:05.739640 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:05.644830 31713 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:05.750111 25707 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
W20260709 12:20:05.757100 31710 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
W20260709 12:20:05.757864 31724 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:20:05.762919 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2494) took 1208 ms. Trace:
W20260709 12:20:05.763288 31725 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:20:05.763545 25632 rpcz_store.cc:276] 0709 12:20:04.554239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.554428 (+   189us) service_pool.cc:224] Handling call
0709 12:20:04.554785 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.679885 (+125100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.762539 (+1082654us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400362754. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:05.762833 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254393600}
I20260709 12:20:05.777464 25751 leader_election.cc:400] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:20:05.777799 25751 leader_election.cc:403] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:20:05.779722 31726 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.773613 31534 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:05.770056 31694 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.453s	user 0.003s	sys 0.007s
I20260709 12:20:05.780234 31716 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:05.768146 31507 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:05.805459 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:05.817838 31421 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:05.818392 31421 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:05.852258 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:05.770330 31693 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:05.870247 31709 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.183s	user 0.006s	sys 0.006s
I20260709 12:20:05.871054 31497 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.918705 31497 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:05.871676 31725 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:20:05.780244 31726 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.812672 31535 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.883634 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:05.883996 31699 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:05.884948 31731 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:05.887917 31648 consensus_meta.cc:287] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.823s	user 0.009s	sys 0.005s
I20260709 12:20:05.888252 31421 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:05.889207 31710 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:20:05.890134 31698 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.427s	user 0.007s	sys 0.003s
I20260709 12:20:05.892102 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:05.950654 25787 raft_consensus.cc:3060] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.893157 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.893692 31707 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.952788 31707 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:05.955406 31680 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.059s	user 0.002s	sys 0.001s
I20260709 12:20:05.955945 31707 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:05.896356 31640 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.594s	user 0.015s	sys 0.005s
I20260709 12:20:05.956727 31640 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:05.904799 31636 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:05.957527 31636 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:05.958478 31636 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.907501 31632 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:05.765880 31713 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:05.966373 31713 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:05.908515 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2501) took 1284 ms. Trace:
I20260709 12:20:05.908972 31695 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:05.948684 31731 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:05.967376 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1348) took 1013 ms. Trace:
I20260709 12:20:05.967780 26335 rpcz_store.cc:276] 0709 12:20:04.953697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.972610 (+ 18913us) service_pool.cc:224] Handling call
0709 12:20:04.977271 (+  4661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:05.193349 (+216078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:05.193386 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:05.193434 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:05.193467 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.967132 (+773665us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400563454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:05.967184 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:05.967316 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496392576}
I20260709 12:20:05.969208 31731 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:05.969969 26341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1125) took 1376 ms (client timeout 1660 ms). Trace:
W20260709 12:20:05.970389 26341 rpcz_store.cc:269] 0709 12:20:04.593915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.600209 (+  6294us) service_pool.cc:224] Handling call
0709 12:20:05.969735 (+1369526us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54004970d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:05.969890 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067131776}
I20260709 12:20:05.970978 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:05.967046 25637 rpcz_store.cc:276] 0709 12:20:04.624146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.624253 (+   107us) service_pool.cc:224] Handling call
0709 12:20:04.624679 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:04.630263 (+  5584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:04.630309 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:04.630934 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 12:20:04.896995 (+266061us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:04.897047 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.897130 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:05.908244 (+1011114us) raft_consensus.cc:1685] finished
0709 12:20:05.908272 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:05.908428 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5076,"raft.queue_time_us":8156,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6572,"thread_start_us":244426,"threads_started":1,"wal-append.queue_time_us":523450}
I20260709 12:20:05.972397 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:05.973085 25786 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.973994 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1128) took 1379 ms. Trace:
I20260709 12:20:05.909950 31665 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:05.869361 31730 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
W20260709 12:20:05.912482 31409 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.531s	user 0.017s	sys 0.001s
I20260709 12:20:05.913887 25713 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:05.915856 31619 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:05.930074 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.976581 31726 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:05.934949 25778 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:05.983266 31535 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:05.960391 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:05.930254 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:05.976755 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.987201 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:05.988085 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:05.990931 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.962855 31421 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:05.994819 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:06.000406 25636 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:06.004962 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.005663 26335 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:05.974391 26343 rpcz_store.cc:276] 0709 12:20:04.594819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.598594 (+  3775us) service_pool.cc:224] Handling call
0709 12:20:05.973790 (+1375196us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54000a37d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:05.973934 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912881024}
I20260709 12:20:06.009828 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:05.969712 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2315) took 1295 ms. Trace:
I20260709 12:20:06.015004 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:06.015355 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.016197 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.028609 26341 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 19: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:20:06.011560 31680 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.055s	user 0.001s	sys 0.001s
I20260709 12:20:06.009970 26343 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:06.032718 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.013974 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:06.044720 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.016078 25631 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:06.012355 25784 rpcz_store.cc:276] 0709 12:20:04.674516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.674616 (+   100us) service_pool.cc:224] Handling call
0709 12:20:04.674934 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:04.988317 (+313383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:04.988354 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:05.001760 (+ 13406us) log.cc:844] Serialized 52 byte log entry
0709 12:20:05.136119 (+134359us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:05.155431 (+ 19312us) log.cc:844] Serialized 14 byte log entry
0709 12:20:05.159672 (+  4241us) log.cc:844] Serialized 14 byte log entry
0709 12:20:05.159948 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.160012 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:05.969492 (+809480us) raft_consensus.cc:1685] finished
0709 12:20:05.969518 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:05.969652 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14745,"prepare.queue_time_us":124336,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":7133,"raft.queue_time_us":46006,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":24452,"thread_start_us":397968,"threads_started":2,"wal-append.queue_time_us":252354}
I20260709 12:20:06.012460 31535 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:06.043278 31707 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.025344 31646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.362s	user 0.053s	sys 0.072s
I20260709 12:20:06.061565 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:05.951771 31648 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.080462 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2845) took 1063 ms. Trace:
I20260709 12:20:06.075030 25778 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:06.081094 25784 rpcz_store.cc:276] 0709 12:20:05.016989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.052496 (+1035507us) service_pool.cc:224] Handling call
0709 12:20:06.079730 (+ 27234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.080004 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.080043 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:06.080106 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:06.080154 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.080242 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.080385 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:20:06.059029 31703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.015s	sys 0.049s
I20260709 12:20:06.087292 25786 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:06.044242 31497 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.054572 31240 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:20:06.058620 31636 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:06.100986 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:06.063890 31699 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.023281 31730 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:20:06.104220 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:06.105674 26341 raft_consensus.cc:2393] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 19: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:06.090579 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2846) took 1073 ms. Trace:
W20260709 12:20:06.083019 31703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.015s	sys 0.049s
I20260709 12:20:06.054221 31622 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.093825 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.106312 25778 rpcz_store.cc:276] 0709 12:20:05.017260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.089586 (+1072326us) service_pool.cc:224] Handling call
0709 12:20:06.089929 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.090150 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.090190 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:06.090248 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:06.090292 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.090363 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.090512 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:06.109022 26336 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:06.106866 26343 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:20:06.110452 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.113582 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.115945 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.103255 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:06.119230 25784 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
W20260709 12:20:06.097229 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 303866us
I20260709 12:20:06.064133 31713 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.063048 31409 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:06.063644 31620 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.067247 31646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.419s	user 0.053s	sys 0.075s
I20260709 12:20:06.122020 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2848) took 1098 ms. Trace:
I20260709 12:20:06.091893 31592 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:06.134620 25603 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:06.139058 31027 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:20:06.119912 25636 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:06.140008 25778 rpcz_store.cc:276] 0709 12:20:05.023457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.108015 (+1084558us) service_pool.cc:224] Handling call
0709 12:20:06.108395 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.121611 (+ 13216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.121649 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:06.121704 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:06.121748 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.121827 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.121955 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:06.153215 25603 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:06.089433 31707 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.115171 31535 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:06.164891 25778 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:06.156339 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.207760 26341 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:06.154412 31645 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:06.172576 31714 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:06.173133 31027 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:06.164129 31459 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:06.136636 31695 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.157807 31592 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.142905 31713 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.119583 31622 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:06.177238 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 74073.4us
I20260709 12:20:06.093951 31731 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:20:06.148255 31734 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:06.180389 26343 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:06.187678 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.191793 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2847) took 1174 ms. Trace:
I20260709 12:20:06.257299 25784 rpcz_store.cc:276] 0709 12:20:05.017567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.088901 (+1071334us) service_pool.cc:224] Handling call
0709 12:20:06.109140 (+ 20239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.137534 (+ 28394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.191491 (+ 53957us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b5400087ad4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:06.191717 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15064832}
I20260709 12:20:06.198884 31703 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.199946 31730 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:06.200532 31697 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:06.206190 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:06.215225 26342 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:06.139225 31620 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.266841 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.267524 25778 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:06.268042 31620 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.268345 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.268723 31620 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:06.224740 31742 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 12:20:06.225368 31736 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:06.271147 26340 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:06.214064 31645 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.227357 31594 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.120129 31699 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.213749 31535 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:06.158008 31630 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.151989 31726 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:06.236768 31459 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.236143 31648 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.263445 26336 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:06.264974 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.283891 25784 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:06.272321 31739 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.284693 31726 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.285593 31592 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.285452 31739 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:06.286783 25785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2310) took 1619 ms (client timeout 1672 ms). Trace:
I20260709 12:20:06.287179 31726 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.287786 31739 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.273311 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.288574 31699 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.290022 31713 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:06.287213 25785 rpcz_store.cc:269] 0709 12:20:04.667242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.667349 (+   107us) service_pool.cc:224] Handling call
0709 12:20:06.286571 (+1619222us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54001a63d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:06.286690 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931024256}
I20260709 12:20:06.291033 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.279281 31535 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.292239 31622 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.274741 31645 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:06.290947 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.270561 31730 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.295650 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1303) took 1839 ms. Trace:
I20260709 12:20:06.296114 25635 rpcz_store.cc:276] 0709 12:20:04.455849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.455998 (+   149us) service_pool.cc:224] Handling call
0709 12:20:04.484278 (+ 28280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:04.706864 (+222586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.295425 (+1588561us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400363154. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:06.295596 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500334464}
I20260709 12:20:06.297147 31730 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.238023 31695 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.285471 31225 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:06.298789 31225 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.300311 31695 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.301000 31225 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.272998 31697 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.303037 31697 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.256950 31731 consensus_queue.cc:1243] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:20:06.270853 31703 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:06.304107 31745 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.132s	user 0.003s	sys 0.000s
I20260709 12:20:06.304215 31535 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.305860 31703 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.260541 31734 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:06.306294 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.306537 31734 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:06.307984 25781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2841) took 1292 ms. Trace:
I20260709 12:20:06.309544 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2844) took 1292 ms. Trace:
I20260709 12:20:06.235013 31707 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.303755 31239 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:06.310969 25784 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:20:06.233235 31716 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 10. Current term is 11.
I20260709 12:20:06.310187 25778 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:20:06.269289 31620 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.316834 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.332719 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:06.329466 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.334151 25636 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:20:06.308394 25781 rpcz_store.cc:276] 0709 12:20:05.015751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.423170 (+407419us) service_pool.cc:224] Handling call
0709 12:20:05.423557 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:05.633171 (+209614us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:05.657934 (+ 24763us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.280361 (+622427us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.301212 (+ 20851us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b4800250158. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29ba8b8aa1 00007f29ba8afb37 00007f29ba8af9ca 00007f29ba9a726f 00007f29ba9a6f37 00007f29ba9bb55c 00007f29ba9bb508 00007f29ba9bb488 00007f29ba9bb443 00007f29ba9ba6ec 00007f29b98e4a96
0709 12:20:06.305993 (+  4781us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.307541 (+  1548us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:06.307585 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.307657 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:06.307705 (+    48us) raft_consensus.cc:1685] finished
0709 12:20:06.307801 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.307923 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41200,"prepare.queue_time_us":30154,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":79906,"raft.run_cpu_time_us":2492,"raft.run_wall_time_us":843391,"spinlock_wait_cycles":8145792,"thread_start_us":566031,"threads_started":3,"wal-append.queue_time_us":460098}
I20260709 12:20:06.309940 25782 rpcz_store.cc:276] 0709 12:20:05.016702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.013130 (+996428us) service_pool.cc:224] Handling call
0709 12:20:06.038953 (+ 25823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.039327 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.039354 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.039797 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.222791 (+182994us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:06.222838 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.222913 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:06.309327 (+ 86414us) raft_consensus.cc:1685] finished
0709 12:20:06.309351 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.309483 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5920,"raft.queue_time_us":3245,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":10000,"thread_start_us":134765,"threads_started":1,"wal-append.queue_time_us":202292}
I20260709 12:20:06.338901 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.316866 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1085) took 1054 ms. Trace:
I20260709 12:20:06.341850 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:06.339116 26341 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:06.342396 25787 raft_consensus.cc:2468] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:06.347918 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.348392 31699 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:06.328557 31689 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.347263 26339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1129) took 1752 ms (client timeout 1741 ms). Trace:
W20260709 12:20:06.355029 26339 rpcz_store.cc:269] 0709 12:20:04.595091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.595227 (+   136us) service_pool.cc:224] Handling call
0709 12:20:06.347056 (+1751829us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b540061c454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:06.347198 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816807680}
W20260709 12:20:06.342715 31743 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.154s	user 0.002s	sys 0.000s
I20260709 12:20:06.348382 25786 rpcz_store.cc:276] 0709 12:20:05.262667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.134477 (+871810us) service_pool.cc:224] Handling call
0709 12:20:06.135560 (+  1083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.135813 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:20:06.160326 (+ 24513us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.315404 (+155078us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:06.315463 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:06.315511 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.316377 (+   866us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.316804 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24357,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":13663,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":3943,"thread_start_us":154227,"threads_started":1}
I20260709 12:20:06.353588 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.337548 31385 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:06.351892 26344 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:06.343078 31713 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:06.339486 31648 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.344628 31535 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.359628 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.347008 31592 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:06.356205 31695 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:06.362634 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.364590 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.360508 31630 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:06.361268 31713 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:06.369050 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.363360 31592 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.376051 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.377002 25787 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:20:06.377394 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.377557 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.378165 25785 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:06.378635 31592 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:06.379518 31620 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.381393 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.386900 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.388502 25635 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:20:06.388875 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.391932 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.357800 31689 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.398175 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.398974 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.376355 31713 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:06.401712 25704 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:20:06.391876 31695 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:06.362960 31648 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.403908 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2313) took 1733 ms. Trace:
I20260709 12:20:06.406303 31459 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.398301 31738 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:20:06.393661 31746 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:06.419030 31689 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.396481 31510 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.023s
W20260709 12:20:06.420222 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.424494 25780 rpcz_store.cc:276] 0709 12:20:04.670570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.670699 (+   129us) service_pool.cc:224] Handling call
0709 12:20:06.403703 (+1733004us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54001248d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:06.403845 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456184704}
I20260709 12:20:06.428721 31595 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:06.431303 26341 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:06.435890 31421 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:06.439502 26335 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:06.427174 31651 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:06.452533 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.444425 31648 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.426944 31027 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:20:06.453722 31630 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
W20260709 12:20:06.461365 26309 leader_election.cc:336] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.863s (SENT)
I20260709 12:20:06.464207 25676 leader_election.cc:400] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:20:06.464743 25676 leader_election.cc:403] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:20:06.431012 31734 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.444890 31459 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:20:06.436651 31645 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.443296 31746 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.454140 31748 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:06.471475 31696 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:20:06.485487 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56156 (request call id 1131) took 1877 ms. Trace:
W20260709 12:20:06.507114 26309 leader_election.cc:336] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.909s (SENT)
I20260709 12:20:06.507697 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:06.486035 26344 rpcz_store.cc:276] 0709 12:20:04.595682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.601375 (+  5693us) service_pool.cc:224] Handling call
0709 12:20:06.349129 (+1747754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.358722 (+  9593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.472840 (+114118us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b540061c450. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:06.472970 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4018636672}
I20260709 12:20:06.466953 31055 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:06.554004 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 75615.8us
W20260709 12:20:06.568670 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:06.604533 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 97476.8us
I20260709 12:20:06.602744 31055 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.608983 31055 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:06.611785 31762 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:20:06.613782 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2837) took 1599 ms. Trace:
I20260709 12:20:06.614168 25783 rpcz_store.cc:276] 0709 12:20:05.014465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.060382 (+ 45917us) service_pool.cc:224] Handling call
0709 12:20:05.060745 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:05.074627 (+ 13882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:05.074674 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.421636 (+1346962us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b2c0007afc8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:20:06.422133 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.462637 (+ 40504us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:06.472555 (+  9918us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.472759 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.472837 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:06.613434 (+140597us) raft_consensus.cc:1685] finished
0709 12:20:06.613545 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.613714 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23598,"prepare.queue_time_us":11026,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":27893,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":156,"spinlock_wait_cycles":101554688,"thread_start_us":34113,"threads_started":1,"wal-append.queue_time_us":29756}
I20260709 12:20:06.615738 31645 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.617045 31697 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:06.617592 31697 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.619000 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:06.619633 31697 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.621438 31535 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:06.621989 31535 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:06.604089 31385 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:20:06.623447 31751 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:06.624356 31594 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:06.624755 31594 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:06.548324 31739 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:06.625701 31594 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.630017 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.631201 31594 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.632200 31535 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.604168 31703 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:06.654687 31703 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.655089 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.656368 25638 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:06.629449 26339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:06.658352 26339 raft_consensus.cc:606] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:06.603806 25677 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:06.625718 31739 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.632936 31592 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.635593 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.456699 31752 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:20:06.454257 31651 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.641229 25705 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:20:06.626621 31746 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.642902 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.676738 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2690) took 1447 ms. Trace:
W20260709 12:20:06.643170 26309 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.034s (SENT)
I20260709 12:20:06.677399 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:06.643860 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:06.654667 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.679582 25632 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:06.658759 26339 raft_consensus.cc:614] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:06.551612 31689 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:06.662910 31734 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:06.663725 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59976.9us
I20260709 12:20:06.667176 31703 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.660815 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:06.667701 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:06.669265 31761 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:20:06.677119 25634 rpcz_store.cc:276] 0709 12:20:05.229706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.324093 (+ 94387us) service_pool.cc:224] Handling call
0709 12:20:05.765169 (+441076us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:05.773026 (+  7857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:05.773068 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:05.773667 (+   599us) log.cc:844] Serialized 52 byte log entry
0709 12:20:06.003402 (+229735us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b1800128200. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:20:06.102991 (+ 99589us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:06.149160 (+ 46169us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.153745 (+  4585us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.159188 (+  5443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.159469 (+   281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:06.228940 (+ 69471us) raft_consensus.cc:1685] finished
0709 12:20:06.228985 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.297507 (+ 68522us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b1800128200. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:20:06.313756 (+ 16249us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b1800128200. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:20:06.676580 (+362824us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54003007d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:06.676691 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14878,"prepare.queue_time_us":106326,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":6464,"raft.queue_time_us":22962,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":66164,"spinlock_wait_cycles":1113974912,"thread_start_us":99313,"threads_started":1,"wal-append.queue_time_us":391351}
I20260709 12:20:06.687057 25713 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:06.673166 31696 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:06.688704 31636 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:20:06.692204 31592 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.692444 31651 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.690416 31646 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.693262 31646 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.693629 31646 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.695168 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:06.699090 31707 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:06.699510 31707 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.624626 31713 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.666432 31695 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:06.699858 31707 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:06.706657 26309 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.108s (SENT)
I20260709 12:20:06.709779 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:06.711342 25713 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
W20260709 12:20:06.713330 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 256649us
I20260709 12:20:06.712767 25603 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:06.717989 25704 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:06.718677 31713 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.719925 26339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:06.720736 26339 raft_consensus.cc:606] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:06.721168 26339 raft_consensus.cc:614] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:06.723034 31497 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:06.723616 31497 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:06.724928 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2843) took 1708 ms. Trace:
I20260709 12:20:06.728098 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:06.729643 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:06.730168 25632 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:06.730584 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:06.734926 26339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:06.735762 26339 raft_consensus.cc:606] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:06.736199 26339 raft_consensus.cc:614] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:06.725558 25779 rpcz_store.cc:276] 0709 12:20:05.016302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.473908 (+457606us) service_pool.cc:224] Handling call
0709 12:20:05.474211 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:05.474409 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:20:05.483397 (+  8988us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.616350 (+1132953us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b180049a560. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a 00007f29b5999a2a
0709 12:20:06.716740 (+100390us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.724472 (+  7732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:06.724547 (+    75us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:06.724612 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.724702 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.724854 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17284,"prepare.queue_time_us":9186,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":62,"raft.queue_time_us":1470,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":13950848,"thread_start_us":98224,"threads_started":1}
I20260709 12:20:06.714826 31636 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:06.708825 31664 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:20:06.743322 25713 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
W20260709 12:20:06.743770 31706 log.cc:927] Time spent T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.423s	user 0.008s	sys 0.000s
I20260709 12:20:06.743949 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:20:06.733722 31592 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:20:06.743793 31739 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:06.745656 31497 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.763667 31027 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:20:06.771580 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:06.776504 31027 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:20:06.777082 31027 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:06.778824 31630 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:06.778278 31027 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.788796 31027 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:06.775911 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:06.794471 31664 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:06.806991 31664 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.809950 31664 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.810667 31646 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.813570 31696 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:06.813685 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:06.814312 25630 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.815479 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:06.803889 31544 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:06.816386 31544 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.819741 31642 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:06.824465 25676 leader_election.cc:400] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20.
I20260709 12:20:06.824905 25676 leader_election.cc:403] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:06.825572 31642 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:06.825938 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.826647 25778 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:06.829862 26343 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:20:06.830983 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.831375 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:06.831629 26339 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:20:06.831748 25779 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:06.831013 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.833272 25676 leader_election.cc:400] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17.
I20260709 12:20:06.833607 25676 leader_election.cc:403] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:06.834180 31695 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17.
I20260709 12:20:06.834748 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:06.835542 31695 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:20:06.835635 26337 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:06.835951 31695 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:06.836414 31695 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:06.837417 31751 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:06.838013 31751 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.838843 31702 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:06.839939 31702 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:06.840267 31702 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:06.840394 31751 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.841746 26344 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:06.836644 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:06.842177 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:06.846007 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.850741 25630 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:06.859347 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:06.870827 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:06.871819 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:06.864503 25779 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 8.
I20260709 12:20:06.876340 25676 leader_election.cc:400] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19.
I20260709 12:20:06.879117 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:06.818414 31544 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:06.835829 31672 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:06.886795 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:06.888108 25709 raft_consensus.cc:2368] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16.
I20260709 12:20:06.889627 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:06.890004 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.890347 25710 raft_consensus.cc:2368] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15.
I20260709 12:20:06.891101 25707 raft_consensus.cc:2410] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:20:06.892385 26344 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:06.872937 31636 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:06.868336 31055 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:06.878757 31642 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20.
I20260709 12:20:06.894615 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:06.913043 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.914233 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.873188 31604 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:06.841161 31702 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.915179 31604 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.917791 31604 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.873347 31739 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:06.919142 31739 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:06.919782 31739 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.919200 31055 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.882246 31421 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:06.922852 31646 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.884473 31696 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:20:06.893973 31459 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:20:06.925345 31459 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:06.925716 31459 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:06.878686 31664 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:06.926921 31664 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:06.929796 31664 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:06.923882 31636 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:06.941334 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:06.941397 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:06.908910 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.895973 31748 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:06.897203 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:06.875175 31699 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:06.952978 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:06.954031 25784 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:06.956881 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:06.957623 26339 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 19: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:06.958654 31459 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.960357 31459 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:06.960942 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:06.937562 31636 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:06.962611 26341 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
W20260709 12:20:06.964254 31721 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 1.495s	user 0.009s	sys 0.004s
I20260709 12:20:06.965736 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2505) took 1768 ms. Trace:
I20260709 12:20:06.966163 25629 rpcz_store.cc:276] 0709 12:20:05.196987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.197163 (+   176us) service_pool.cc:224] Handling call
0709 12:20:05.197537 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:05.200465 (+  2928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:05.200502 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:05.221048 (+ 20546us) log.cc:844] Serialized 26 byte log entry
0709 12:20:05.311957 (+ 90909us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:05.312010 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.312094 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:06.963617 (+1651523us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:06.965403 (+  1786us) raft_consensus.cc:1685] finished
0709 12:20:06.965431 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:06.965670 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2961,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14895,"spinlock_wait_cycles":4490496,"thread_start_us":48861,"threads_started":1,"wal-append.queue_time_us":187493}
I20260709 12:20:06.967906 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:06.968734 25708 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:06.970361 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.976691 31510 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:06.887877 31672 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:06.981549 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.982293 25712 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:06.985248 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.988279 25632 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:06.962170 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:06.989107 25704 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:20:07.000083 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:07.002827 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:07.003835 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.005510 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.005770 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.006640 25779 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:06.922308 31055 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.008368 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.013586 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.014194 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:07.014991 26341 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:20:07.015726 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.016683 25779 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:07.017297 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.018281 25707 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:07.018739 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.022253 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.023073 25783 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:07.025986 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.028517 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:07.029670 25708 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:07.030900 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:07.035772 25782 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: [16.14-18.15]
I20260709 12:20:07.040436 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:07.041146 26339 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:07.050375 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:07.051461 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:07.053401 25785 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:06.949745 31748 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:07.059937 25783 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:20:07.060101 25708 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:07.062426 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.070822 31381 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:07.071242 31381 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:07.072007 31381 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.073107 31381 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.076017 25785 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:07.077800 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:07.078326 25778 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.085108 25778 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
W20260709 12:20:07.088388 31760 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.630s	user 0.003s	sys 0.001s
I20260709 12:20:07.095510 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:07.097473 26339 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
W20260709 12:20:07.108177 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.098712 25709 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:07.114755 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:07.098803 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:07.098428 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.100636 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:07.096609 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:07.101238 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.057677 31497 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:07.104244 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.104954 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:07.064860 31748 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:07.014658 31544 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:20:07.133263 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:07.135118 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:07.137121 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:07.148051 26307 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.148070 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.152611 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:07.152686 25711 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.156877 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.157868 25707 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:07.158990 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.163929 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:07.161877 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:07.167534 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69245.7us
I20260709 12:20:07.101516 31055 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.171706 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:07.172413 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:07.106232 31728 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:07.168285 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.175213 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.174247 31728 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:07.177271 31699 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:20:07.177524 31775 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.142s	user 0.004s	sys 0.000s
I20260709 12:20:07.177754 31699 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.177776 31728 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.179397 31728 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.183001 31544 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:07.183162 31716 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:07.183578 31716 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.183806 31544 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:07.183995 31716 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:07.184207 31544 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:07.185067 31544 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.186077 31544 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:07.191459 31768 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:20:07.191759 31015 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:07.135190 31497 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:07.197986 31497 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:07.174595 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.206771 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:07.211629 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.213774 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.171891 31636 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:07.178447 31699 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.230734 31696 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:20:07.231192 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.231635 31696 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:20:07.245471 25677 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:07.167258 31385 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:07.168630 31646 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:07.256664 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.262100 26335 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.168941 31509 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:07.264918 31387 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:07.267725 31509 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:07.234098 31746 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.243613 31254 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:07.273677 25704 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:07.265208 31630 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:07.263407 31385 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.304665 31385 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.209636 31055 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.308590 31055 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:07.262705 31636 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:07.311385 31636 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.301693 31509 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:07.302889 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:07.317401 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.269095 31646 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:07.292865 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:07.319897 25707 pending_rounds.cc:85] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 9
I20260709 12:20:07.322568 25707 pending_rounds.cc:107] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:20:07.322923 25707 raft_consensus.cc:2889] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:07.333038 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 169851us
I20260709 12:20:07.333797 26307 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:07.320338 31778 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:20:07.291697 31727 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.296579 31254 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:07.325662 31646 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.329010 31728 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:07.393401 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48907.1us
W20260709 12:20:07.345005 31707 consensus_meta.cc:287] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.006s
I20260709 12:20:07.332469 31510 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:07.358713 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 194853us
I20260709 12:20:07.405146 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:07.406668 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:07.346534 31630 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:07.427130 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.357965 31254 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.437073 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.358101 31535 raft_consensus.cc:2804] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:07.395998 31728 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:07.345782 31695 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.446448 31689 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.357501 31727 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.396608 31699 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.467615 25705 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:07.469470 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:07.475123 25784 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.003s
I20260709 12:20:07.457809 31672 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.477173 25629 raft_consensus.cc:1275] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:07.488662 25784 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:07.408823 31707 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.472409 31728 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:07.501895 25712 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
W20260709 12:20:07.506690 31773 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:20:07.508144 31707 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:07.508819 31784 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.510006 31695 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.407632 31722 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:20:07.502461 25705 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:20:07.409909 31748 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.516867 31748 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.517705 25711 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
W20260709 12:20:07.520923 31777 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.484s	user 0.003s	sys 0.000s
I20260709 12:20:07.523201 31544 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:07.423604 31510 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.527439 25708 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:07.473248 31535 raft_consensus.cc:697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:07.479571 31726 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:20:07.540644 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 277293us
W20260709 12:20:07.533838 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 343045us
I20260709 12:20:07.548229 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:07.549316 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:07.532135 31651 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:20:07.538228 31387 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:07.565280 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.564769 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.548136 31726 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:07.570887 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.544086 31774 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 12:20:07.567119 31739 consensus_meta.cc:287] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.001s
I20260709 12:20:07.567958 31714 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:07.544942 31716 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:07.590112 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 326453us
I20260709 12:20:07.617669 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:07.592347 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.627458 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:07.572783 31027 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.580755 31738 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:07.547646 31630 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.625613 31714 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:07.632673 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 305039us
I20260709 12:20:07.622566 31640 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.655182 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:07.632045 31756 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.584874 31739 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.582319 31497 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.641234 31738 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:07.639894 31381 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:07.663893 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:07.666661 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 174857us
I20260709 12:20:07.652977 31714 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:07.672111 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:07.675630 25779 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:07.671319 31391 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:07.676795 31727 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:07.677393 31727 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:07.654884 31741 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:07.680105 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:07.688872 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:07.689639 25632 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.677922 31727 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.664151 31790 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.668434 31748 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:07.666563 31738 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:07.629735 31535 consensus_queue.cc:237] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.674409 31699 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:07.709533 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:07.661388 31640 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.664212 31055 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:20:07.712401 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 305811us
I20260709 12:20:07.692268 31713 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:20:07.664228 31756 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.658461 31726 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.657198 31387 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.672528 31381 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.693372 31707 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:20:07.765583 31771 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.729s	user 0.000s	sys 0.003s
I20260709 12:20:07.766925 25713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3189) took 1004 ms. Trace:
W20260709 12:20:07.768216 31782 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:20:07.767345 25713 rpcz_store.cc:276] 0709 12:20:06.762409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.762962 (+   553us) service_pool.cc:224] Handling call
0709 12:20:06.765510 (+  2548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.772112 (+  6602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.772144 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.775122 (+  2978us) log.cc:844] Serialized 26 byte log entry
0709 12:20:07.024511 (+249389us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:07.024573 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.024773 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.765022 (+740249us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:07.766573 (+  1551us) raft_consensus.cc:1685] finished
0709 12:20:07.766598 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.766857 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9258,"raft.queue_time_us":6210,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":2475,"spinlock_wait_cycles":68644096,"thread_start_us":239051,"threads_started":1,"wal-append.queue_time_us":261337}
W20260709 12:20:07.770031 31792 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:07.771845 31739 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.769954 31508 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:07.751556 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:07.759518 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 322524us
W20260709 12:20:07.755025 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 102172us
I20260709 12:20:07.692029 31741 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.786414 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:07.794463 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.779610 31497 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.800776 31748 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:07.755892 31699 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:07.721688 31510 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.788147 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.819336 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:07.782846 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64668.4us
I20260709 12:20:07.719936 25632 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
W20260709 12:20:07.816293 31793 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:20:07.804431 31787 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:20:07.715184 31795 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:07.719570 31713 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.785830 31508 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:07.827960 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:07.826289 26308 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:07.829030 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 156990us
I20260709 12:20:07.848193 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:20:07.867273 31385 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.002s
I20260709 12:20:07.867990 31385 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.849077 31713 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:07.871230 31640 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:07.871474 31798 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:20:07.873123 31716 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.858700 31722 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.826944 31604 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:20:07.887022 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.881800 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.885918 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:07.889434 25784 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:07.826097 31779 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:20:07.852839 31508 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.875182 31722 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.903971 25784 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:07.911324 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:07.905544 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:07.856775 31497 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:07.905253 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:07.906023 31728 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.971084 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:07.972039 25780 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.919826 31794 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:07.886633 31604 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:07.910144 31620 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:20:07.904176 31806 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.830971 31636 consensus_queue.cc:1048] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:20:07.932498 31381 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.867297 31727 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:07.894080 31716 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:07.847697 31646 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.910826 31648 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:20:07.979928 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 152094us
I20260709 12:20:07.912122 31731 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:07.848305 31254 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:07.934567 31741 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:07.988173 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 108042us
I20260709 12:20:07.917335 31510 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:08.010541 26338 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:07.994004 25784 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:20:07.996520 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 170310us
I20260709 12:20:07.979944 31604 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.969183 31497 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:07.997709 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:07.978397 31751 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.973738 31809 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:20:07.989305 31716 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:07.984455 31027 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:20:08.019243 26344 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
W20260709 12:20:08.014228 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 182967us
W20260709 12:20:08.026672 25751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.501s (SENT) [suppressed 33 similar messages]
I20260709 12:20:08.007640 31648 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:07.979254 31794 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.032970 25713 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:20:08.018085 31497 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:08.040387 25713 pending_rounds.cc:85] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 14
I20260709 12:20:08.042104 25713 pending_rounds.cc:107] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:20:08.042375 25713 raft_consensus.cc:2889] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:08.026872 31716 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:08.021380 31809 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:08.007429 31731 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:07.989262 31806 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.064779 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 1102) took 1004 ms. Trace:
I20260709 12:20:08.068418 25779 rpcz_store.cc:276] 0709 12:20:07.059916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.061972 (+  2056us) service_pool.cc:224] Handling call
0709 12:20:08.064533 (+1002561us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b5400191254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:08.064700 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408254848}
I20260709 12:20:08.065490 25786 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.015336 31510 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.060783 31810 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:08.039656 31818 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.026602 31751 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.089875 25786 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:20:08.075376 31636 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:08.067744 31804 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 12:20:08.061342 31702 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:08.061720 31756 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:08.062000 31809 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:20:08.133809 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:08.134851 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.137136 25709 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:08.138895 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:08.141865 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:08.142561 25712 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
W20260709 12:20:08.144236 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 117002us
I20260709 12:20:08.127517 31027 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:08.149302 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:08.081688 31713 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.156384 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:08.160065 26341 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:08.160357 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:08.122673 31636 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.166435 25780 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:08.191401 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:08.201292 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:08.201812 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45421.5us
I20260709 12:20:08.202816 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:08.203547 25780 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:08.203842 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:08.204653 26341 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:08.091248 31818 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.209596 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:08.081337 31810 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.154153 31728 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:08.214618 31689 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:08.214855 31784 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.166371 31731 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:08.178682 31646 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.165144 31821 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:20:08.151981 31509 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:20:08.128041 31648 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.223865 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:08.221859 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:08.222368 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.170804 31722 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.231653 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:08.239720 25779 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.184617 31751 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.183622 31806 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:08.233191 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:08.145938 31707 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:20:08.215399 31421 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:20:08.232321 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 234708us
I20260709 12:20:08.162424 31713 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.174649 31794 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.220938 31385 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.185575 31699 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:20:08.252750 25634 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:08.227010 31821 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:08.254442 25635 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:20:08.170203 31508 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.225688 31689 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.224109 31642 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:20:08.254020 26341 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
W20260709 12:20:08.274307 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:08.277292 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44173.1us
I20260709 12:20:08.269886 31689 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:08.244025 31604 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.283389 31604 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.285233 31818 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.225926 31784 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.287848 31784 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.216070 31738 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.289916 31738 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.291404 25630 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:08.293819 25636 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:08.295185 31510 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:08.296518 31714 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.000s
I20260709 12:20:08.296954 31714 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.300442 31716 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.301994 25783 raft_consensus.cc:2368] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13.
I20260709 12:20:08.302914 31788 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:08.303359 31788 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.303702 31788 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:08.304672 31672 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:08.307039 31716 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.309020 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:08.309865 25780 raft_consensus.cc:2410] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:20:08.311862 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.313509 25637 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:08.324507 25779 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:08.327200 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:08.380115 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76807.4us
I20260709 12:20:08.382587 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:08.382824 25779 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:20:08.388468 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 108118us
I20260709 12:20:08.388814 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.389278 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:08.389747 26341 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:08.313115 31421 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:08.326602 31834 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.394922 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:08.395744 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 155071us
W20260709 12:20:08.398851 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 71745.7us
W20260709 12:20:08.465835 31790 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.000s
I20260709 12:20:08.466732 31620 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.503744 31815 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:08.503887 31620 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.504474 31815 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.467762 31689 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.468406 31713 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:08.507735 31820 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:20:08.508149 31713 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.469375 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.469743 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.511493 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.512332 26341 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:08.470252 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:08.470644 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.513134 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:08.514011 26341 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:08.471125 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:08.514906 26340 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:08.472836 31816 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:08.510375 31713 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.506258 31620 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.518905 31816 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.477532 31544 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:08.478845 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.480176 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.524503 31387 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:08.525072 31387 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.481819 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.527045 31387 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.486063 25633 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:08.487321 31421 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.383993 31728 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.463253 31818 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:08.530864 31728 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.392261 31642 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:08.493458 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.513152 25712 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:08.534827 25778 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:08.494656 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:08.457898 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 68712.2us
I20260709 12:20:08.398191 31827 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.537001 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.456284 31834 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.537643 25638 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:20:08.537828 26340 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:08.457548 31385 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:20:08.532236 31790 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:08.536829 31510 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.530274 31497 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.578570 31510 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.568084 25603 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:08.560599 25677 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:08.554049 31714 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.583806 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.579228 31544 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:08.568702 31385 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:08.622702 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 54725.5us
I20260709 12:20:08.624507 31772 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:08.624549 31015 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:08.640386 31385 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.625530 31838 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:20:08.622143 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:08.544956 31809 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.629915 25631 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:08.582654 31620 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:08.623749 31592 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:20:08.546687 31827 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.644058 31772 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:08.658041 25632 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:20:08.632848 31831 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:08.651362 31592 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.631498 31738 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:20:08.700313 31619 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:08.701146 31728 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:08.700984 31619 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.703297 31619 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.703572 31827 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.644827 31642 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.699712 31592 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:08.710071 25633 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:08.719830 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.668710 31831 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.736375 31689 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.741019 31619 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:08.741703 31619 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.742182 31816 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:08.744004 31619 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.745823 25630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2537) took 1072 ms. Trace:
I20260709 12:20:08.748315 31810 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:08.714349 31835 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:20:08.751042 31497 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.652208 31620 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.752576 31636 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:08.753751 31620 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.754961 31254 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.757325 31834 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.751112 31815 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.763338 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:08.689877 31696 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:08.763993 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.764469 31696 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.767246 25787 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:08.767794 31831 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:08.768049 31839 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.147s	user 0.000s	sys 0.002s
I20260709 12:20:08.767140 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.768757 25633 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:08.754123 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2957) took 1485 ms. Trace:
I20260709 12:20:08.769990 25707 rpcz_store.cc:276] 0709 12:20:07.268825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.274162 (+  5337us) service_pool.cc:224] Handling call
0709 12:20:07.300005 (+ 25843us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:07.358276 (+ 58271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:07.358329 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:07.359185 (+   856us) log.cc:844] Serialized 52 byte log entry
0709 12:20:07.640573 (+281388us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:07.640621 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.753571 (+1112950us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b54003c91d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:08.753637 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:08.753705 (+    68us) raft_consensus.cc:1685] finished
0709 12:20:08.753726 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:08.754045 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2664,"raft.queue_time_us":36394,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":13124,"spinlock_wait_cycles":45056768,"thread_start_us":201664,"threads_started":1,"wal-append.queue_time_us":316322}
I20260709 12:20:08.774988 31696 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.746441 25630 rpcz_store.cc:276] 0709 12:20:07.673693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.675333 (+  1640us) service_pool.cc:224] Handling call
0709 12:20:07.698582 (+ 23249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:08.291819 (+593237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.745473 (+453654us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54002acbd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:08.745744 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1362928512}
I20260709 12:20:08.788326 25783 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:08.793139 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:08.794219 25783 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:08.795058 31510 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:08.802294 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.763820 31809 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.815744 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:08.812656 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.814123 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.815024 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:08.809118 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 155444us
I20260709 12:20:08.852599 25710 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:08.855546 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.856386 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:08.857910 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.859599 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:08.860071 31636 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:08.862643 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:08.863446 25783 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:08.803176 31696 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:08.868428 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:08.869218 26338 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
W20260709 12:20:08.873721 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.874032 25712 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:20:08.874838 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:08.878894 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:08.859450 31695 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:08.896366 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.880939 31510 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:20:08.904749 31619 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.905390 31619 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.873216 31696 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.908425 31619 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.909003 31696 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:08.910676 31788 consensus_meta.cc:287] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.003s
I20260709 12:20:08.911327 31788 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:08.909260 31821 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.913975 31788 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:08.914112 31821 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:08.886752 31757 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:08.867332 31731 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:20:08.926785 25711 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:20:08.868604 31636 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.929594 31757 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:08.923209 31848 log.cc:927] Time spent T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
I20260709 12:20:08.925855 31748 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:08.946678 25783 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:08.891176 31695 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.930123 31254 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.935534 31757 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:08.955969 26335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1364) took 2309 ms. Trace:
I20260709 12:20:08.956477 26335 rpcz_store.cc:276] 0709 12:20:06.641671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.641899 (+   228us) service_pool.cc:224] Handling call
0709 12:20:06.642280 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:07.267088 (+624808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.950925 (+1683837us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54006df454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:08.951123 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425004928}
W20260709 12:20:08.957942 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 335928us
I20260709 12:20:08.959954 31672 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:08.961143 31672 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:08.961544 31672 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:08.964800 25706 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:08.966408 25708 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:08.976661 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.976397 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.976286 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.984486 25677 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:08.987267 25633 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:08.976075 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.973855 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.975718 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:08.992609 25637 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:08.994444 25707 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:08.995636 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.999869 26335 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:09.001821 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.005034 25783 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:09.009299 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:08.952351 31254 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:08.966979 31731 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:09.008205 31788 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:08.947440 31651 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:20:09.010995 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:08.949292 31816 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:20:09.012523 31421 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:09.056690 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:09.057438 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:09.058063 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:09.012570 31741 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:20:09.039330 31713 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:08.996748 31510 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.953815 31810 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:20:09.031860 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:09.057766 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:08.949545 31748 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.111326 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:08.978180 31850 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.976140 31664 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:09.121762 31385 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:09.126991 31385 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:09.127454 31385 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:09.127832 31254 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:09.111701 31831 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.106839 31421 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:09.091090 25638 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:09.032025 31648 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:09.093464 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:09.141920 25705 raft_consensus.cc:2368] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
I20260709 12:20:09.018512 31731 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:09.005765 31620 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:20:09.142947 25713 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:09.145190 25752 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.146678 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:09.031500 31672 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:09.137633 31856 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:20:09.108150 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:09.004060 31784 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 12:20:09.109999 31636 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:09.030943 31696 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:20:09.008857 31642 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:20:09.114022 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:08.974352 31832 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:09.121145 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 89355us
I20260709 12:20:09.185283 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:09.139698 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:09.134315 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 1103) took 2038 ms (client timeout 1634 ms). Trace:
W20260709 12:20:09.186319 25778 rpcz_store.cc:269] 0709 12:20:07.096451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.096573 (+   122us) service_pool.cc:224] Handling call
0709 12:20:09.134104 (+2037531us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001a7cd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:09.134234 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3233239424}
W20260709 12:20:09.137070 31854 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:20:09.152165 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.124091 31664 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:09.154975 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 102081us
I20260709 12:20:09.171438 31728 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:09.130468 31592 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:09.198928 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.198738 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:09.199801 25705 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:09.147521 25713 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
W20260709 12:20:09.105679 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 146475us
W20260709 12:20:09.166921 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 163157us
I20260709 12:20:09.133723 31421 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:09.202546 26308 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.132604 31831 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:09.101282 31788 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.123512 31510 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.180540 31672 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:09.226089 31739 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:09.226533 31739 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:09.227443 31790 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:20:09.227945 31790 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:09.227398 31739 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.228741 31790 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.230017 31790 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:20:09.214741 25710 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:20:09.171903 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75290.2us
I20260709 12:20:09.204618 25629 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: [16.11-18.12]
I20260709 12:20:09.204905 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.205199 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.201792 31595 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:09.201843 31695 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:09.196316 31689 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.211326 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.211395 31863 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:09.263645 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.213984 31592 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:09.213833 31815 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:09.126278 31850 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.267396 25632 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:20:09.268002 31850 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:09.268841 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.221257 31510 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:09.222726 31728 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:09.222110 31857 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:20:09.269480 25632 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:09.270299 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.201687 31459 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:20:09.191808 31832 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.241005 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:09.241662 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.243189 31664 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.247385 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.247818 25707 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:20:09.208125 31731 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.258080 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.263645 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:09.270627 31689 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.234664 25706 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:09.271477 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:09.287997 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.385092 25636 raft_consensus.cc:2368] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:09.387308 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:09.389623 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 118231us
I20260709 12:20:09.400043 25633 raft_consensus.cc:2393] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 9: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:09.269094 31651 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:20:09.403028 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 250926us
I20260709 12:20:09.251175 31595 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.405460 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.272167 31648 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.408087 25629 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:09.408202 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:09.272440 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61321.1us
I20260709 12:20:09.405794 25638 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:09.409245 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.384997 31254 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:09.415612 25633 raft_consensus.cc:2368] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:09.410212 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:09.422777 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.430851 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.436770 25785 raft_consensus.cc:2410] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:20:09.431227 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:09.431883 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.384207 31592 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:09.433065 31664 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:09.441164 25784 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:09.245122 31385 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.413563 31595 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:09.446741 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.450834 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.450299 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.451826 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.409915 31651 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:09.454805 25779 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:09.444309 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.456657 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.440696 31592 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:09.481279 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.483453 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.484153 25780 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:09.484932 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:09.485416 25780 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:09.437952 31648 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.486454 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:09.496170 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:09.496775 25782 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:09.473126 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:09.503250 25779 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:09.504522 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.505436 25638 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:09.507189 31741 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:09.507763 31741 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.510093 31741 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:09.511122 25778 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:20:09.512260 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:09.512799 25778 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:09.445958 31510 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:09.434902 31862 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:20:09.474349 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.519634 31862 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:09.520190 31862 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:09.520823 31786 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:09.520643 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.522409 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.475833 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:09.523162 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.523809 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:09.524389 25637 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:20:09.531350 31864 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:09.532975 31832 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:09.532927 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.536021 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.506400 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:09.538786 25638 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:09.452811 31731 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:09.546312 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.546463 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.547807 25638 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:20:09.540567 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:09.541226 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 59847.3us
I20260709 12:20:09.569963 25782 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:09.571553 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:09.572229 25782 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:20:09.572782 25601 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.574107 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:09.576025 25780 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:09.576126 25638 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:09.558007 26307 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.580744 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:09.586382 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:09.587498 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.588366 25780 raft_consensus.cc:2410] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:20:09.584311 31821 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:20:09.543391 31788 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:09.562942 31510 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:09.563651 31832 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:09.567685 31838 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:09.593989 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:09.543638 31254 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:09.589908 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.610061 25637 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:09.580408 31865 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.582104 31741 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.617014 25779 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:09.550976 31636 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:20:09.617297 25705 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:09.613608 31832 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:09.592491 31831 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:20:09.610061 25782 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 9.
I20260709 12:20:09.602581 31604 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:09.623575 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.597712 31707 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:09.641072 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.585112 31786 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 12:20:09.634419 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.623478 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.596745 31774 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:20:09.591285 31510 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:09.688016 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.683655 25782 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
W20260709 12:20:09.706991 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.597018 31592 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:09.717005 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 93788us
I20260709 12:20:09.715343 31739 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:09.694475 31871 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.626241 31865 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:09.722185 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.722707 31818 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:09.693218 31421 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.724892 31421 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:09.725394 31862 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.726982 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.725848 25779 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:09.728806 25712 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:09.730012 31871 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.731629 31862 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:09.730023 31818 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:09.737926 31818 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:09.741298 31385 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:09.720883 31508 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:09.742430 31592 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:09.733920 31865 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:09.767107 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 188505us
I20260709 12:20:09.744724 31748 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:09.765672 31672 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.540s	user 0.014s	sys 0.001s
I20260709 12:20:09.633126 31741 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.752899 31774 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:20:09.605790 31781 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:09.779891 25630 raft_consensus.cc:2410] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:20:09.769101 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:09.749882 31015 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:20:09.812731 25778 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:09.771102 31651 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.747239 31849 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:09.822968 31875 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:09.779510 31871 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:09.829687 25782 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:20:09.832537 31790 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.602s	user 0.004s	sys 0.012s
I20260709 12:20:09.810534 31509 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 12:20:09.844895 31852 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.740s	user 0.006s	sys 0.007s
I20260709 12:20:09.771327 31391 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:20:09.807142 31722 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:09.863462 25677 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:09.868466 31862 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:09.622455 31254 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.193s
I20260709 12:20:09.630447 31838 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.186s
I20260709 12:20:09.879905 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:09.880370 26337 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:09.884461 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:09.816416 31757 consensus_meta.cc:287] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.852s	user 0.002s	sys 0.015s
I20260709 12:20:09.884735 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:09.889179 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.891079 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:09.914521 25782 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:09.816742 31781 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:09.808187 31604 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:09.904794 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.908416 25632 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:09.907119 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:09.923542 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:09.887912 31880 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:09.910449 31790 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:20:09.884088 31849 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:09.957868 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.907704 31815 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 12:20:09.961906 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:10.066639 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.055225 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 1014) took 1560 ms. Trace:
I20260709 12:20:10.054170 31722 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:09.932260 31832 consensus_queue.cc:1243] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:09.933699 31838 consensus_queue.cc:1243] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:20:09.938680 31672 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:10.067922 25704 rpcz_store.cc:276] 0709 12:20:08.494462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:08.494599 (+   137us) service_pool.cc:224] Handling call
0709 12:20:08.494908 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:08.531188 (+ 36280us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:08.531218 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:08.531668 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 12:20:08.873438 (+341770us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:08.873476 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.873539 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:09.844235 (+970696us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:09.937772 (+ 93537us) raft_consensus.cc:1685] finished
0709 12:20:09.937802 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.055128 (+117326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38259,"raft.queue_time_us":37584,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":7163,"spinlock_wait_cycles":8192768,"thread_start_us":341470,"threads_started":1,"wal-append.queue_time_us":488225}
I20260709 12:20:09.911406 31862 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:09.932896 31757 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.063336 31790 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:10.062821 31707 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:10.054309 31880 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.098589 31672 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:10.142800 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 218486us
W20260709 12:20:10.145749 25751 connection.cc:256] RPC call timeout handler was delayed by -1.5564646430000266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 12:20:10.157459 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:10.142117 31774 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:10.109642 31707 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.100939 31604 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.100497 31781 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.153472 31878 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.163904 25635 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:20:10.170572 25751 connection.cc:256] RPC call timeout handler was delayed by -1.3168623850001495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2638s for a response
I20260709 12:20:10.172510 25710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2973) took 1074 ms. Trace:
I20260709 12:20:10.175202 25752 leader_election.cc:400] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
I20260709 12:20:10.175866 25752 leader_election.cc:403] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:10.175158 25710 rpcz_store.cc:276] 0709 12:20:09.098346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.113722 (+ 15376us) service_pool.cc:224] Handling call
0709 12:20:10.172247 (+1058525us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b5400147b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:10.172435 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80490880}
I20260709 12:20:10.181382 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.183823 25710 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:10.184013 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:10.185154 25704 raft_consensus.cc:2368] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:10.186556 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.187564 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.188596 25710 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:10.203444 25635 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
W20260709 12:20:10.206099 31595 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.000s
I20260709 12:20:10.215525 31739 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:20:10.215484 31595 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:10.209544 31731 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.635s	user 0.007s	sys 0.007s
I20260709 12:20:10.211294 31886 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.227492 25780 raft_consensus.cc:2410] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:20:10.228294 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51522 (request call id 2563) took 1236 ms. Trace:
I20260709 12:20:10.236430 25635 rpcz_store.cc:276] 0709 12:20:08.991503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.157096 (+1165593us) service_pool.cc:224] Handling call
0709 12:20:10.228225 (+ 71129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:10.252624 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.253845 25635 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:10.193785 31871 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.256563 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:10.256906 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:10.240888 31881 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 12:20:10.262969 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.235688 31784 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.267015 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:10.235628 31891 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.271266 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1365) took 1040 ms. Trace:
I20260709 12:20:10.167948 31790 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.281056 25634 raft_consensus.cc:1275] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:10.281878 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1359) took 1307 ms. Trace:
I20260709 12:20:10.282277 25634 rpcz_store.cc:276] 0709 12:20:08.973928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.264448 (+1290520us) service_pool.cc:224] Handling call
0709 12:20:10.280737 (+ 16289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.281555 (+   818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.281809 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:10.284765 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:10.286875 25751 connection.cc:256] RPC call timeout handler was delayed by -1.2687617770000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
I20260709 12:20:10.288883 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.290849 31870 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:10.191352 31672 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.192541 31878 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.263521 31889 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.262054 31421 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:10.271023 31862 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.298530 25635 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 10.
I20260709 12:20:10.312994 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:10.313251 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:10.291646 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.292055 31728 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.303201 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2720) took 1042 ms. Trace:
W20260709 12:20:10.304441 31894 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:20:10.305855 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 130812us
I20260709 12:20:10.262804 31871 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.269971 31849 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.314592 25632 rpcz_store.cc:276] 0709 12:20:09.260506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.270089 (+  9583us) service_pool.cc:224] Handling call
0709 12:20:10.303022 (+1032933us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54001d79d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:10.303138 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466165632}
I20260709 12:20:10.258240 31774 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.280623 31891 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.314765 31876 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.342958 31876 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.345144 31876 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.280211 31784 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.314355 31728 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:10.317003 31877 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:20:10.195308 31772 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.241999 31886 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.318406 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.295010 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:10.319342 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:10.346091 31707 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.395036 26344 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.346349 31648 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.398181 31891 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.398908 31897 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:10.399603 31728 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.347726 31027 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:20:10.337167 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.405385 25783 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:10.349156 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.319288 31870 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:20:10.314882 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.408174 25783 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:20:10.350636 31630 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:10.355159 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1364) took 1139 ms. Trace:
W20260709 12:20:10.355906 31882 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 12:20:10.357703 31592 consensus_meta.cc:287] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.002s
I20260709 12:20:10.360666 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57966 (request call id 2968) took 1268 ms. Trace:
I20260709 12:20:10.417063 25707 rpcz_store.cc:276] 0709 12:20:09.092555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.092765 (+   210us) service_pool.cc:224] Handling call
0709 12:20:10.360407 (+1267642us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400203d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:10.360564 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287839488}
I20260709 12:20:10.297114 31757 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.337359 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1363) took 1121 ms. Trace:
I20260709 12:20:10.419306 25637 rpcz_store.cc:276] 0709 12:20:09.215552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.624618 (+409066us) service_pool.cc:224] Handling call
0709 12:20:09.625124 (+   506us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:20:10.091916 (+466792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.091967 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:20:10.103035 (+ 11068us) log.cc:844] Serialized 78 byte log entry
0709 12:20:10.278458 (+175423us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:10.278529 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.278616 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:10.337154 (+ 58538us) raft_consensus.cc:1685] finished
0709 12:20:10.337174 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.337309 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59223,"raft.queue_time_us":72357,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4446,"thread_start_us":532000,"threads_started":2,"wal-append.queue_time_us":210933}
I20260709 12:20:10.421898 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:10.339116 31885 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:20:10.293664 31254 consensus_queue.cc:1048] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:20:10.274142 25633 rpcz_store.cc:276] 0709 12:20:09.230945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.629408 (+398463us) service_pool.cc:224] Handling call
0709 12:20:09.629637 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.629815 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:10.058551 (+428736us) log.cc:844] Serialized 14 byte log entry
0709 12:20:10.261773 (+203222us) log.cc:844] Serialized 14 byte log entry
0709 12:20:10.270945 (+  9172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:10.270995 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:10.271040 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.271103 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.271196 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8807,"prepare.queue_time_us":524831,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":851,"raft.queue_time_us":118053,"spinlock_wait_cycles":9787648,"thread_start_us":486518,"threads_started":2,"wal-append.queue_time_us":3963}
W20260709 12:20:10.380586 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.318143 31391 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:20:10.428840 31391 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:10.429751 31619 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.315063 31889 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.432996 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.433301 31889 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.319736 31421 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:10.434674 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.337715 31880 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.328397 31878 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.330561 31595 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.367179 31821 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.430145 31391 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.436918 31821 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.402463 31731 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:10.439113 31821 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.391206 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:20:10.440598 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 49538.7us
I20260709 12:20:10.442102 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.444620 26340 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:20:10.444816 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:10.445750 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:10.395465 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76207.8us
I20260709 12:20:10.446846 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:10.447098 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:10.448108 26340 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:10.448523 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.448805 31870 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:10.449394 31757 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:10.449572 26340 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:20:10.449900 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:10.395318 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 76981.5us
I20260709 12:20:10.452029 26340 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:20:10.350363 31866 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.405143 31027 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:10.452806 31866 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.453581 31027 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.453995 25752 leader_election.cc:400] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:10.454697 31027 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:10.454918 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.455288 31866 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.383291 31748 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.455755 26339 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:10.456246 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 2977) took 1312 ms. Trace:
I20260709 12:20:10.456101 31748 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.456631 25712 rpcz_store.cc:276] 0709 12:20:09.144023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.234996 (+ 90973us) service_pool.cc:224] Handling call
0709 12:20:09.235267 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.729865 (+494598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.456054 (+726189us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400223dd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:10.456197 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1134522240}
I20260709 12:20:10.388005 31739 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14.
I20260709 12:20:10.410862 31630 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:10.458459 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.414644 26344 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:10.459054 31748 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.458917 31739 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:10.407553 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:10.460021 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52598.3us
I20260709 12:20:10.454515 25752 leader_election.cc:403] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:20:10.461476 31731 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:10.461696 31849 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:10.462337 31849 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.416087 31871 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:10.404318 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.451990 31886 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.457111 26342 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:10.408854 25638 rpcz_store.cc:276] 0709 12:20:09.216183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.630014 (+413831us) service_pool.cc:224] Handling call
0709 12:20:09.634344 (+  4330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:10.104828 (+470484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.104869 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.108750 (+  3881us) log.cc:844] Serialized 26 byte log entry
0709 12:20:10.156652 (+ 47902us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:10.156693 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.156768 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:10.354923 (+198155us) raft_consensus.cc:1685] finished
0709 12:20:10.354947 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.355094 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140056,"raft.queue_time_us":248831,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3104,"thread_start_us":377548,"threads_started":2,"wal-append.queue_time_us":70794}
I20260709 12:20:10.392133 31876 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.460112 31739 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:10.422106 31784 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:10.412358 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:10.440675 31254 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:20:10.462466 25536 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 304) took 29923 ms (client timeout 15000 ms). Trace:
I20260709 12:20:10.462749 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:10.464129 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.463668 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:10.435765 25633 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 11: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:10.464313 25635 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:10.472569 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.466169 25750 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:10.497786 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:10.468209 26335 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:20:10.498777 25637 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:10.500376 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:10.501792 25751 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:10.470583 25783 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:10.503126 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:10.505909 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2566) took 1052 ms. Trace:
I20260709 12:20:10.506297 25635 rpcz_store.cc:276] 0709 12:20:09.453089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.505181 (+1052092us) service_pool.cc:224] Handling call
0709 12:20:10.505462 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.505624 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.505651 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:10.505691 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:10.505724 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.505776 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.505862 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:10.506449 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.507853 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2569) took 1052 ms. Trace:
I20260709 12:20:10.508188 25635 rpcz_store.cc:276] 0709 12:20:09.454951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.507189 (+1052238us) service_pool.cc:224] Handling call
0709 12:20:10.507438 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.507593 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.507615 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:10.507652 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:10.507679 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.507725 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.507806 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:10.509243 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2567) took 1055 ms. Trace:
I20260709 12:20:10.509579 25637 rpcz_store.cc:276] 0709 12:20:09.453713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.508561 (+1054848us) service_pool.cc:224] Handling call
0709 12:20:10.508820 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.508976 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.508998 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:10.509035 (+    37us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:10.509066 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.509111 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.509195 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:10.511651 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1367) took 1033 ms. Trace:
I20260709 12:20:10.512050 25637 rpcz_store.cc:276] 0709 12:20:09.477854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.510658 (+1032804us) service_pool.cc:224] Handling call
0709 12:20:10.511044 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.511273 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.511307 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:10.511360 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:10.511406 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.511473 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.511589 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:10.516330 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.518067 25712 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:10.519996 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:10.520720 25712 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.458274 31630 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.462836 31849 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:10.504478 31897 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:10.462903 31731 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:10.541658 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.547727 26335 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:10.472740 31748 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.546893 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.553313 25632 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
W20260709 12:20:10.539678 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 108446us
I20260709 12:20:10.467006 31739 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:10.525352 31757 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:10.503912 31254 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:10.525219 31870 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:20:10.493021 25536 rpcz_store.cc:269] 0709 12:19:40.539538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.555871 (+ 16333us) service_pool.cc:224] Handling call
0709 12:19:41.187771 (+631900us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:10.462399 (+29274628us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:01.042458 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:01.043276 (+   818us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:01.043330 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:20:01.043350 (+    20us) tablet.cc:662] Decoding operations
0709 12:20:01.045006 (+  1656us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:01.045133 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:01.045162 (+    29us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:20:01.177812 (+132650us) tablet.cc:701] Row locks acquired
0709 12:20:01.177871 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:20:01.178074 (+   203us) write_op.cc:270] Start()
0709 12:20:01.178283 (+   209us) write_op.cc:276] Timestamp: P: 1783599601178007 usec, L: 0
0709 12:20:01.178336 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 12:20:01.179249 (+   913us) log.cc:844] Serialized 6857 byte log entry
0709 12:20:01.483473 (+304224us) op_driver.cc:464] REPLICATION: finished
0709 12:20:01.724295 (+240822us) write_op.cc:301] APPLY: starting
0709 12:20:01.724427 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:01.838267 (+113840us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:01.838433 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:02.529658 (+691225us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b5c000e460c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bbb81331 00007f29bbb81221 00007f29bbb7bf6c 00007f29b92dc30b 00007f29b92dbfd3 00007f29b93dae34 00007f29b93c697c 00007f29b93cad61 00007f29b93cad19 00007f29b93caca9 00007f29b93cac71
0709 12:20:02.740372 (+210714us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:02.740775 (+   403us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 12:20:02.740841 (+    66us) write_op.cc:312] APPLY: finished
0709 12:20:02.741354 (+   513us) log.cc:844] Serialized 156 byte log entry
0709 12:20:03.028566 (+287212us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:03.028781 (+   215us) write_op.cc:454] Released schema lock
0709 12:20:03.028874 (+    93us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:03.046549 (+ 17675us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:03.052976 (+  6427us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3030067,"reported_tablets":48,"spinlock_wait_cycles":109143296,"child_traces":[["op",{"apply.queue_time_us":240383,"apply.run_cpu_time_us":134267,"apply.run_wall_time_us":1305066,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75289,"num_ops":17,"prepare.queue_time_us":218728,"prepare.run_cpu_time_us":23559,"prepare.run_wall_time_us":256708,"raft.queue_time_us":85409,"raft.run_cpu_time_us":34529,"raft.run_wall_time_us":118538,"replication_time_us":304943,"spinlock_wait_cycles":8249216,"thread_start_us":615701,"threads_started":5,"wal-append.queue_time_us":593882,"wal-append.run_cpu_time_us":18579,"wal-append.run_wall_time_us":2204701}]]}
I20260709 12:20:10.561357 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:10.525774 31728 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:10.507360 31871 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:10.547988 31897 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:10.526616 31865 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.526866 31898 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:10.537369 31630 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:10.472462 31664 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 12:20:10.466801 31876 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.574214 31870 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:10.556740 31774 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:10.578018 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61789.1us
I20260709 12:20:10.581043 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.537793 31784 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.435712 31421 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.616475 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.617545 31672 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:10.618669 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:10.619874 31672 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:10.620868 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.621978 31672 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.622439 31672 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.622871 31672 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.583240 25712 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:10.472740 31619 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.584918 25676 leader_election.cc:400] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13.
I20260709 12:20:10.536971 31896 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 12:20:10.629071 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44281.9us
I20260709 12:20:10.629675 31896 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:10.630230 31896 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:10.630257 31619 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.631265 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.631227 31896 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.632755 31896 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.632930 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.633610 25710 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:10.575564 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.566922 31748 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.634694 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.635491 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.635673 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:10.574919 31254 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:10.636119 25707 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:20:10.636482 25709 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 10: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:10.636147 31748 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:10.639500 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.640151 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.640728 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.611590 31728 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.634424 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:10.642062 31728 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.642791 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:10.644435 31728 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:10.572309 31731 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.606014 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:10.645479 31731 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.647612 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:10.648277 31731 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:10.649047 31731 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:10.650370 31731 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:10.652740 25782 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:10.654423 25778 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:10.661100 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.662791 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.663610 25778 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 10.
I20260709 12:20:10.667986 31254 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.669864 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.584043 31757 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:10.674352 25778 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 8. Current term is 9. Ops: []
I20260709 12:20:10.587208 31897 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:10.587409 31790 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:10.681952 25631 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:10.595579 31870 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.677071 31889 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:10.586233 31880 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.094s
W20260709 12:20:10.685526 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 116491us
I20260709 12:20:10.684557 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:10.707476 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67406.9us
I20260709 12:20:10.613458 31913 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:10.587194 31865 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.675272 31784 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.712754 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.724579 26336 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:20:10.727340 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.730513 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.723351 31619 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:10.732642 31911 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:20:10.733942 31865 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:10.697701 31689 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.613286 31774 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.737784 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.738744 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.612983 31739 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.743692 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.743779 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:10.715780 31790 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:10.729698 31784 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.713441 31728 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:20:10.692436 31917 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 12:20:10.767642 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 127832us
I20260709 12:20:10.744441 31689 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.776834 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:10.781745 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:10.750414 31664 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:10.732669 31619 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.779400 31890 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:10.781227 31689 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.819619 25785 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:10.795135 31917 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:10.794379 31595 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.804926 31924 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:10.805359 31876 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:10.816587 31905 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:20:10.758255 31757 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:10.819558 31920 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.106s	user 0.002s	sys 0.000s
I20260709 12:20:10.788823 31664 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.804281 31592 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 12:20:10.836802 31906 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.154s	user 0.000s	sys 0.002s
I20260709 12:20:10.824254 31917 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:10.845681 31907 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 12:20:10.825417 31421 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:20:10.808740 31790 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:10.902581 31254 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:10.907312 31875 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:10.998857 31726 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.013s	sys 0.020s
W20260709 12:20:11.011267 31726 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.013s	sys 0.020s
W20260709 12:20:11.018868 31933 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:20:11.027266 31619 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:11.009820 31927 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:20:11.001722 31917 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.045849 31917 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:11.052289 31741 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.053689 31689 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:11.060038 31664 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.061285 31774 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.002848 31875 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.071386 31672 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:10.968649 31849 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.106842 31849 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.109081 31672 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.111387 31875 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:11.119488 31630 consensus_meta.cc:287] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.001s
I20260709 12:20:11.292873 25635 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:11.293232 25632 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:20:11.407238 31672 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:11.294334 25638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.000s	sys 0.000s
W20260709 12:20:11.296192 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.002s	sys 0.000s
W20260709 12:20:11.308336 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.381645 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.384469 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.390902 31728 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.391185 31713 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:11.391503 31254 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.392014 31731 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:11.124939 31898 consensus_meta.cc:287] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 12:20:11.423065 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.423602 31898 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:11.418856 31713 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:11.423866 25638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.002s	sys 0.000s
I20260709 12:20:11.182533 31897 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.424935 31731 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.426190 31713 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.428277 31897 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:11.429040 25785 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:11.190670 31784 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.426292 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2568) took 1971 ms. Trace:
I20260709 12:20:11.432251 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.432920 31784 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:11.398116 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.431196 25638 rpcz_store.cc:276] 0709 12:20:09.454338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.506855 (+1052517us) service_pool.cc:224] Handling call
0709 12:20:10.507178 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:10.538072 (+ 30894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.538104 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.548690 (+ 10586us) log.cc:844] Serialized 26 byte log entry
0709 12:20:11.425231 (+876541us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:11.425421 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.425601 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.425728 (+   127us) raft_consensus.cc:1685] finished
0709 12:20:11.425755 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.426117 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":528,"raft.queue_time_us":33616,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":9815,"spinlock_wait_cycles":13696,"thread_start_us":837701,"threads_started":1,"wal-append.queue_time_us":338013}
I20260709 12:20:11.434729 31728 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:11.418987 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.002s	sys 0.000s
I20260709 12:20:11.428458 31713 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:11.439069 25638 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
W20260709 12:20:11.439531 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 649081us
I20260709 12:20:11.160403 31870 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.415378 31672 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.450867 31741 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:11.439424 31254 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.452241 31741 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.453853 31592 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.446739 31027 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:11.446110 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2575) took 1537 ms. Trace:
I20260709 12:20:11.456084 25638 rpcz_store.cc:276] 0709 12:20:09.906185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.437481 (+1531296us) service_pool.cc:224] Handling call
0709 12:20:11.438682 (+  1201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.443454 (+  4772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.443697 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:11.442665 31731 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.459640 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:11.454613 31592 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.450363 31870 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.457208 31027 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.458200 31672 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.462740 31254 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:11.478654 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:11.479713 26336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 1038) took 1033 ms. Trace:
I20260709 12:20:11.480296 26336 rpcz_store.cc:276] 0709 12:20:10.445627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.448737 (+  3110us) service_pool.cc:224] Handling call
0709 12:20:10.449025 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.726854 (+277829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.477317 (+750463us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540044b454. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.479339 (+  2022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633233152}
I20260709 12:20:11.483476 31027 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:11.476821 25601 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.005s	sys 0.009s
I20260709 12:20:11.489696 31741 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.491379 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:11.492599 25601 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.006s	sys 0.009s
I20260709 12:20:11.527221 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2875) took 2043 ms. Trace:
W20260709 12:20:11.527904 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 751824us
I20260709 12:20:11.527740 25780 rpcz_store.cc:276] 0709 12:20:09.483962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.589671 (+105709us) service_pool.cc:224] Handling call
0709 12:20:11.526668 (+1936997us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b54000eea54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:11.526986 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1402589440}
I20260709 12:20:11.513586 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1366) took 2040 ms. Trace:
I20260709 12:20:11.507654 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.532387 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2576) took 1622 ms. Trace:
I20260709 12:20:11.522038 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.533274 25638 rpcz_store.cc:276] 0709 12:20:09.906506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.460085 (+1553579us) service_pool.cc:224] Handling call
0709 12:20:11.475014 (+ 14929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.481764 (+  6750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:11.481786 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:11.490101 (+  8315us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:11.490130 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.490171 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.528662 (+ 38491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:11.496117 31592 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.493924 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:11.500205 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2703) took 2294 ms. Trace:
W20260709 12:20:11.492697 25704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.022s	sys 0.019s
I20260709 12:20:11.535663 31917 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:11.536005 25704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.022s	sys 0.019s
I20260709 12:20:11.536259 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.536351 31917 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:11.537356 31917 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:11.537567 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.504076 31728 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.534987 25636 rpcz_store.cc:276] 0709 12:20:09.205914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.575599 (+369685us) service_pool.cc:224] Handling call
0709 12:20:09.583435 (+  7836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.597706 (+ 14271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:09.597774 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:09.597858 (+    84us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:09.597935 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.499479 (+1901544us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54003011d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.499589 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.499917 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31881088}
I20260709 12:20:11.538230 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.540416 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.540580 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:11.508481 25630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60736 (request call id 1357) took 2534 ms (client timeout 3058 ms). Trace:
I20260709 12:20:11.518788 31865 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:11.542748 31865 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:11.543618 26337 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:11.544468 31865 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.545776 31015 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:11.546111 31865 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:11.546830 31015 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.544930 31027 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.542084 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.531828 25635 rpcz_store.cc:276] 0709 12:20:09.473238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.509046 (+1035808us) service_pool.cc:224] Handling call
0709 12:20:10.509285 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.412641 (+903356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.512691 (+100050us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400364a54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.513487 (+   796us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799794944}
W20260709 12:20:11.541857 25630 rpcz_store.cc:269] 0709 12:20:08.973392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:08.973531 (+   139us) service_pool.cc:224] Handling call
0709 12:20:11.507867 (+2534336us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b540020b7d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:11.508182 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814406016}
I20260709 12:20:11.562353 25784 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:11.517674 31850 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:11.572744 31254 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:11.572543 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.574074 25784 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:11.574061 31027 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.576854 31027 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.549690 31728 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.589856 31728 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.590745 25785 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:11.591451 25784 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:11.592128 31728 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:11.592322 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.592695 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:11.593598 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53350.7us
I20260709 12:20:11.572750 31850 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:11.596514 26309 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:11.596616 31850 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.607547 31891 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.608124 31015 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.608520 31891 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.610713 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1376) took 1159 ms. Trace:
I20260709 12:20:11.611136 25638 rpcz_store.cc:276] 0709 12:20:10.451365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.568261 (+1116896us) service_pool.cc:224] Handling call
0709 12:20:11.609783 (+ 41522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.610219 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:11.610269 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:11.610352 (+    83us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:11.610398 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.610474 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.610622 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:11.616079 31896 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:11.617002 31896 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:11.618741 31943 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.406s	user 0.000s	sys 0.007s
I20260709 12:20:11.619860 31896 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.570235 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1375) took 1116 ms. Trace:
I20260709 12:20:11.624578 25780 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:11.632699 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.636020 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:11.638609 31595 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.807s	user 0.009s	sys 0.017s
I20260709 12:20:11.638897 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.639148 31595 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:11.640394 25785 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:20:11.641036 25630 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:11.647334 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.622198 25636 rpcz_store.cc:276] 0709 12:20:10.451111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.538958 (+1087847us) service_pool.cc:224] Handling call
0709 12:20:11.539535 (+   577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:11.548147 (+  8612us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:11.548185 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:11.548931 (+   746us) log.cc:844] Serialized 26 byte log entry
0709 12:20:11.558042 (+  9111us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:11.558081 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.558149 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.561052 (+  2903us) raft_consensus.cc:1685] finished
0709 12:20:11.561071 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.567412 (+  6341us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4439,"raft.queue_time_us":6361,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":301,"thread_start_us":8642,"threads_started":1,"wal-append.queue_time_us":9220}
I20260709 12:20:11.656785 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.658397 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.657764 31254 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:11.669914 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.666946 31850 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.673360 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.681712 25638 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:11.667801 31891 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.670313 31254 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:11.693898 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:11.694614 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:11.695248 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.695910 31859 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:20:11.694444 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 96638.6us
I20260709 12:20:11.696380 31859 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:11.697583 31859 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.705703 25631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1360) took 2731 ms. Trace:
I20260709 12:20:11.687050 31254 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:11.716917 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1377) took 1249 ms. Trace:
I20260709 12:20:11.710736 25631 rpcz_store.cc:276] 0709 12:20:08.974154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.614195 (+1640041us) service_pool.cc:224] Handling call
0709 12:20:10.614589 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.698524 (+ 83935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.705060 (+1006536us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b540018ddd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.705484 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154063744}
I20260709 12:20:11.723387 25780 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:11.725051 25780 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:11.726788 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 130464us
I20260709 12:20:11.731838 31784 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:11.733115 31784 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:11.733278 31772 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:11.733583 31784 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:11.742897 25531 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 8 to 12, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 12 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:11.745217 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.746104 31739 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:11.746786 25781 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:11.752465 25786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1139) took 1577 ms. Trace:
I20260709 12:20:11.715976 31917 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.779039 26336 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:11.770128 31741 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.786254 26343 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:20:11.788307 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.788914 25629 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:11.746402 25531 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 11 to 12, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 12 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:11.791184 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.793309 26340 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:11.793462 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.794817 31508 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.796343 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.780517 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.717851 25635 rpcz_store.cc:276] 0709 12:20:10.467778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.570708 (+1102930us) service_pool.cc:224] Handling call
0709 12:20:11.602176 (+ 31468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.602756 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:11.659016 (+ 56260us) log.cc:844] Serialized 14 byte log entry
0709 12:20:11.691977 (+ 32961us) log.cc:844] Serialized 14 byte log entry
0709 12:20:11.709845 (+ 17868us) log.cc:844] Serialized 14 byte log entry
0709 12:20:11.710299 (+   454us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:11.710357 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:11.716372 (+  6015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.716443 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.716561 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":935,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":32197,"spinlock_wait_cycles":73842432,"thread_start_us":71019,"threads_started":2,"wal-append.queue_time_us":32255}
I20260709 12:20:11.799281 31508 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.802744 31879 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.800779 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.809269 31651 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:11.809123 31865 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:11.810446 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2574) took 1904 ms. Trace:
I20260709 12:20:11.810701 31865 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:11.811213 31865 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:11.811489 31508 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:11.812110 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.812599 31865 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.814159 31865 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:11.814846 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.816452 26344 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:11.815675 25531 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 2 to 8, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 8 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:11.820253 25705 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:20:11.821353 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.824745 31741 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.825712 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:11.735054 31784 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.768762 25786 rpcz_store.cc:276] 0709 12:20:10.174566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.300371 (+125805us) service_pool.cc:224] Handling call
0709 12:20:10.300782 (+   411us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:20:10.400213 (+ 99431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.400260 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:20:10.401195 (+   935us) log.cc:844] Serialized 78 byte log entry
0709 12:20:10.538042 (+136847us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:10.538089 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.751606 (+1213517us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b5400320354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.751694 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.751775 (+    81us) raft_consensus.cc:1685] finished
0709 12:20:11.751795 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.752216 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":227415808,"thread_start_us":136341,"threads_started":1,"wal-append.queue_time_us":189515,"wal-append.run_cpu_time_us":2832,"wal-append.run_wall_time_us":1151831}
W20260709 12:20:11.828002 25629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51520 (request call id 2704) took 2621 ms (client timeout 1931 ms). Trace:
W20260709 12:20:11.828425 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2881) took 2220 ms (client timeout 2095 ms). Trace:
I20260709 12:20:11.829757 25782 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:11.822352 25710 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 10. Current term is 11. Ops: []
I20260709 12:20:11.830616 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.867591 25705 pending_rounds.cc:85] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 9
I20260709 12:20:11.874341 25705 pending_rounds.cc:107] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:20:11.874624 26335 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:11.874752 25705 raft_consensus.cc:2889] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:11.831112 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 1028) took 1200 ms. Trace:
W20260709 12:20:11.876091 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.833920 26338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1393) took 1951 ms. Trace:
I20260709 12:20:11.876852 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.877188 26338 rpcz_store.cc:276] 0709 12:20:09.882548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.883522 (+   974us) service_pool.cc:224] Handling call
0709 12:20:09.883793 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.109746 (+225953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.109800 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:10.109872 (+    72us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:10.109921 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.833508 (+1723587us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400573d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.833571 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.833719 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518924544}
W20260709 12:20:11.836112 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.810956 25632 rpcz_store.cc:276] 0709 12:20:09.905890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.569204 (+663314us) service_pool.cc:224] Handling call
0709 12:20:10.570396 (+  1192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.414280 (+843884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.810180 (+395900us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b540021f054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.810389 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1658962816}
I20260709 12:20:11.880128 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:11.881500 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:11.839771 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 119350us
I20260709 12:20:11.882889 31859 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:11.883930 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.842177 31863 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.885737 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:11.886446 31863 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.887627 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.888839 25707 raft_consensus.cc:2410] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:20:11.889076 31863 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.844753 31917 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.806172 31879 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.892333 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:11.893079 25706 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:11.893550 25633 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:20:11.894807 31728 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:11.895130 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.895543 31784 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.896028 25705 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:11.850528 31699 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.896906 31699 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:11.897316 31509 consensus_queue.cc:1175] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7bf8f20701245bcbaa09cc663025f97" responder_term: 17 status { last_received { term: 17 index: 13 } last_committed_idx: 12 last_received_current_leader { term: 17 index: 13 } }
I20260709 12:20:11.850395 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.899309 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.900025 25631 raft_consensus.cc:2410] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:20:11.900810 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.900960 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.901526 25786 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:11.846757 31900 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.907264 25631 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:11.854959 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:11.855542 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.856683 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.861027 31788 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:11.862196 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.863232 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2885) took 1599 ms. Trace:
W20260709 12:20:11.864357 31942 log.cc:927] Time spent T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.653s	user 0.006s	sys 0.000s
I20260709 12:20:11.865285 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:11.873360 25779 rpcz_store.cc:269] 0709 12:20:09.608313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.623023 (+ 14710us) service_pool.cc:224] Handling call
0709 12:20:11.828008 (+2204985us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b5400099f54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:11.828155 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89097984}
I20260709 12:20:11.896203 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2565) took 2443 ms. Trace:
I20260709 12:20:11.851353 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.902108 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:11.874133 26343 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:11.846719 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.849649 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.903131 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:11.915313 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:11.908489 25783 rpcz_store.cc:276] 0709 12:20:10.263343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.409483 (+146140us) service_pool.cc:224] Handling call
0709 12:20:10.409745 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.502884 (+ 93139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.862739 (+1359855us) spinlock_profiling.cc:243] Waited 60.6 ms on lock 0x7b540011fb54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.863053 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139442560}
W20260709 12:20:11.873179 25629 rpcz_store.cc:269] 0709 12:20:09.206068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.446389 (+240321us) service_pool.cc:224] Handling call
0709 12:20:11.827755 (+2381366us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b54001d7ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:11.827925 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5360706944}
I20260709 12:20:11.875000 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.917395 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:11.914191 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1145) took 1206 ms. Trace:
I20260709 12:20:11.920221 25708 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:11.874680 31722 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.921308 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.921556 31722 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.922827 26340 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:11.923398 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.894217 31879 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.922142 25636 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:11.921972 25708 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:11.925042 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.924211 25787 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:11.915062 31900 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.920230 25784 rpcz_store.cc:276] 0709 12:20:10.708268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.602092 (+893824us) service_pool.cc:224] Handling call
0709 12:20:11.626821 (+ 24729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.652802 (+ 25981us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:11.716012 (+ 63210us) log.cc:844] Serialized 14 byte log entry
0709 12:20:11.913630 (+197618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:11.913726 (+    96us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:11.913811 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.913916 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.914091 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5100,"prepare.queue_time_us":118587,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":8934,"raft.queue_time_us":156,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":189,"spinlock_wait_cycles":26496,"thread_start_us":177066,"threads_started":2,"wal-append.queue_time_us":137629}
I20260709 12:20:11.934572 31728 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:20:11.961107 31713 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.971907 31713 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:11.938534 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:11.875936 25704 rpcz_store.cc:276] 0709 12:20:10.630789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.631692 (+   903us) service_pool.cc:224] Handling call
0709 12:20:10.631981 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.640932 (+  8951us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:20:10.741540 (+100608us) log.cc:844] Serialized 14 byte log entry
0709 12:20:11.570244 (+828704us) log.cc:844] Serialized 14 byte log entry
0709 12:20:11.632275 (+ 62031us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:11.634711 (+  2436us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:11.646976 (+ 12265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.830798 (+183822us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b54002ec7d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.830849 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.831023 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":99728,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":760,"raft.queue_time_us":7368,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":2740,"spinlock_wait_cycles":20082432,"thread_start_us":926287,"threads_started":3,"wal-append.queue_time_us":751163}
I20260709 12:20:11.943188 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.974406 25630 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:11.974705 25705 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:11.975247 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.975647 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.943672 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2375) took 1272 ms. Trace:
I20260709 12:20:11.943885 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:11.976490 25782 rpcz_store.cc:276] 0709 12:20:10.671276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.672032 (+   756us) service_pool.cc:224] Handling call
0709 12:20:11.829069 (+1157037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.873454 (+ 44385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.943422 (+ 69968us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400099f50. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:11.943605 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659965824}
I20260709 12:20:11.977797 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.972404 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.939703 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.948307 25632 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:20:11.946632 31788 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.952536 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.983630 31421 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:11.953915 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.916330 31254 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.985230 25787 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:11.986729 31254 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:11.935096 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.987763 25787 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:11.988159 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.988530 25752 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:11.935746 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:11.989390 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.960019 31630 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.931103 25709 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:11.960651 31791 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:11.960717 31699 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.961199 31741 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.936265 25778 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:11.936640 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:11.935907 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:11.993501 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.995182 31772 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.995764 31772 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:11.935051 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.996547 31015 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:11.964529 26336 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:11.919600 25633 rpcz_store.cc:276] 0709 12:20:09.452444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.506743 (+1054299us) service_pool.cc:224] Handling call
0709 12:20:11.828570 (+1321827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.893989 (+ 65419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.895189 (+  1200us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b54001d7ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:11.895998 (+   809us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001d7ed0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:11.896142 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3186856192}
I20260709 12:20:11.939636 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:11.969799 31722 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.939213 25710 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:20:11.997802 25705 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:12.000850 26343 raft_consensus.cc:1240] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:11.977833 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:11.978788 26340 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:11.945621 25712 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:20:11.984611 25783 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:20:11.984270 31421 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:11.990582 31672 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.990236 31630 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:12.013571 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.007696 31739 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:20:12.014947 31630 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:12.015697 31672 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.017733 31672 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.017946 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.019302 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.019380 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.019829 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:12.020366 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.020587 25783 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:20:12.020970 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.021472 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:12.021793 25705 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:12.022116 25783 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:12.022852 31510 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:12.023352 25783 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:11.960155 31900 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:11.997946 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:11.996601 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:12.024096 25752 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.042428 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:11.999080 25784 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:11.976610 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.000500 25713 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.001572 25782 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:12.010210 31748 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:20:11.997418 31850 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:12.010927 25781 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
W20260709 12:20:12.011276 25778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2383) took 1262 ms (client timeout 1645 ms). Trace:
I20260709 12:20:11.990769 31791 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.007889 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.011556 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.012099 31734 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:12.054656 25778 rpcz_store.cc:269] 0709 12:20:10.749120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.757525 (+  8405us) service_pool.cc:224] Handling call
0709 12:20:12.011058 (+1253533us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000b04d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:12.011210 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2639315712}
I20260709 12:20:12.012573 31728 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:11.997243 31015 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.049233 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:12.051587 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.050240 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.057941 25706 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:12.056226 31739 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.022866 31713 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:11.984618 31788 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.062657 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.065320 25711 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:12.039999 31896 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:12.069044 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:12.049530 31898 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:12.072057 25704 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:12.070880 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.074384 25637 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:12.061868 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.056612 31741 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:12.074942 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:12.083467 25781 raft_consensus.cc:2410] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:20:12.005872 25630 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:12.023300 31510 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:12.074923 25638 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:12.064083 31728 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:12.098157 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.074869 31508 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:12.078732 31772 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:12.075699 31898 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.109845 25705 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:20:12.082876 31741 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.088203 31510 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:12.089370 31699 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:20:12.115527 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 119031us
I20260709 12:20:12.105499 26308 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:12.071673 31850 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.110435 31508 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.146049 25704 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:12.146283 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:12.103650 25709 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:12.090041 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.122529 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:12.123351 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 309267us
I20260709 12:20:12.126577 31821 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.115378 31898 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.128888 31914 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:12.121161 31741 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.131251 26341 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:12.135313 25708 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:12.100486 31917 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:20:12.157424 25707 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
W20260709 12:20:12.159955 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 70028.5us
I20260709 12:20:12.136669 31699 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:12.091327 31734 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.148628 31508 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.140242 31859 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.164614 25752 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.150168 31254 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.159031 31739 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:12.192189 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:12.192066 26340 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:12.177517 31699 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:20:12.205504 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 195843us
I20260709 12:20:12.203329 25782 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:12.177817 31821 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:12.211452 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 154332us
I20260709 12:20:12.204535 31865 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:12.214838 31015 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.215868 31734 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:12.216872 31821 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.227218 31592 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.227484 31791 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:12.234030 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 147683us
I20260709 12:20:12.229022 25677 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:12.162307 31728 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.220407 31739 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:12.242517 31874 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:12.242852 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 80596.5us
I20260709 12:20:12.243088 31874 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.243691 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:12.249259 31672 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:12.250313 31865 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:12.250944 31865 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.252759 31772 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:20:12.253288 31865 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:12.254729 31850 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:12.257409 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:12.258069 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.258090 25778 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:20:12.259732 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.260535 25708 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:12.232709 31748 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:20:12.263515 25637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1370) took 2636 ms. Trace:
W20260709 12:20:12.264339 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.263928 25637 rpcz_store.cc:276] 0709 12:20:09.626986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.538909 (+911923us) service_pool.cc:224] Handling call
0709 12:20:10.539214 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:12.096863 (+1557649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.263231 (+166368us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400362c54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:12.263450 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3527808896}
I20260709 12:20:12.236068 31914 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.265360 31914 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.266386 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:12.267952 31914 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:12.257800 31772 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:12.275038 25713 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:12.276213 25712 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:12.280100 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.280882 25709 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:20:12.281651 31896 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:12.282071 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:12.282600 31961 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.062s	user 0.002s	sys 0.000s
I20260709 12:20:12.282335 31896 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.282842 25781 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:12.283670 25780 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:12.284271 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:12.285200 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.285902 25710 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:12.289027 31970 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:12.289808 31970 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:12.284948 25784 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:12.293890 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.295866 31385 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:12.296018 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52411.2us
I20260709 12:20:12.296729 31385 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.297834 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:12.298905 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.299230 31728 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.299496 31969 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:12.300130 31969 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 17 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:12.300933 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.301126 31969 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.239641 31421 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.302654 31791 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:12.293744 31510 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.306118 31510 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:12.307783 31385 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:12.308965 26341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56146 (request call id 1034) took 1864 ms (client timeout 1514 ms). Trace:
W20260709 12:20:12.309396 26341 rpcz_store.cc:269] 0709 12:20:10.444437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.446594 (+  2157us) service_pool.cc:224] Handling call
0709 12:20:12.308724 (+1862130us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54004812d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:12.308869 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3470485248}
I20260709 12:20:12.310475 31592 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:12.313495 31592 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:12.284948 31896 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:12.314024 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.314967 31385 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:12.315575 31385 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:12.316808 31509 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:12.317098 31385 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:12.319139 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50510.4us
I20260709 12:20:12.320499 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:12.321497 31815 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:12.321767 31385 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:12.321934 31815 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.322338 31815 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.317317 31509 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:12.325273 31509 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.323206 25786 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:12.326113 25713 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:12.326671 25787 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:12.329942 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:12.331818 31963 log.cc:927] Time spent T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:20:12.323983 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:12.332460 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 90300.9us
I20260709 12:20:12.239046 31630 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:20:12.334673 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 175140us
I20260709 12:20:12.335707 25780 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:20:12.322180 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:12.340508 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.343546 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:12.344264 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 65651.4us
I20260709 12:20:12.342082 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:12.347553 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 153489us
I20260709 12:20:12.332252 31874 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:12.359784 31850 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:12.365885 31850 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.360625 25603 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.345708 31509 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.367635 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:12.339607 31630 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:12.368773 31850 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:12.290477 31970 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:20:12.351678 31254 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:12.372390 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.353859 31772 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:12.386183 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:12.385826 31739 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.387938 31970 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:12.393042 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:12.384766 31254 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.393975 31791 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:12.395635 31917 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:12.397339 31672 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.398242 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:12.407076 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:12.416939 31850 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.421204 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.423190 31592 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:12.372498 31821 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:20:12.339399 31815 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:12.436419 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50344.1us
I20260709 12:20:12.438125 31970 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:12.379985 31859 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:20:12.441056 31254 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:12.441754 25707 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:12.456444 25780 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:12.457952 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:12.457798 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 62684.2us
I20260709 12:20:12.474144 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:12.457201 31821 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.443202 31741 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.469539 31713 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:12.475014 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.489671 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:12.490985 26337 raft_consensus.cc:2368] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:20:12.484393 31816 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:12.495343 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:12.496743 31924 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:20:12.497511 31781 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:12.446455 31772 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.508013 25781 raft_consensus.cc:1240] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:12.485134 31784 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:12.483115 31821 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:12.504475 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.489562 31015 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.514513 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.515226 25707 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
W20260709 12:20:12.518371 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60527.9us
I20260709 12:20:12.475569 31672 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.523489 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.525679 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.526938 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:12.530532 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.509994 31781 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.516705 31713 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:12.516556 31816 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:12.541851 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 148916us
I20260709 12:20:12.517521 31917 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:12.537638 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.551069 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:12.552467 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.549871 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:12.513118 31699 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.541153 31781 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.558212 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:12.562053 31850 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:12.557832 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:12.567152 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:12.510499 31630 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:12.592418 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 52954us
W20260709 12:20:12.594857 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57352.3us
I20260709 12:20:12.598019 25709 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:12.598069 25631 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:20:12.602442 25676 leader_election.cc:400] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:20:12.603499 25676 leader_election.cc:403] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:12.605404 31508 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.607909 31508 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.610661 31783 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.611712 31636 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.613385 31741 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:12.614742 31772 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:12.616622 31699 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.622967 31815 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:12.623163 31713 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:12.626988 31971 log.cc:927] Time spent T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:20:12.617656 31970 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.627331 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.628297 31850 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:12.628722 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.628931 31850 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:12.629528 31850 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:12.629817 31831 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:12.630301 31831 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:12.630745 31831 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:12.634604 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.583822 31015 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:12.646811 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49988.9us
W20260709 12:20:12.647851 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 252136us
I20260709 12:20:12.649377 25784 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:12.652884 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:12.611773 31672 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:12.658569 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 132934us
I20260709 12:20:12.659781 31728 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:12.662678 31728 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.663170 31728 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.667403 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:12.666466 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:12.656059 31816 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.654811 31748 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.687146 26338 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:20:12.681685 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.658141 31741 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:12.696628 31672 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:12.697232 31672 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:12.698031 31672 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:12.677047 31982 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:12.703312 31982 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.671530 31898 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:12.712674 25785 raft_consensus.cc:1240] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:20:12.667039 31784 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:12.735144 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.739637 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:12.743992 25677 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:12.753779 25635 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:20:12.755203 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 85376.7us
I20260709 12:20:12.755290 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:12.756659 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 335576us
I20260709 12:20:12.706764 31788 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:12.768805 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.770604 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.771236 25708 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:12.736302 31982 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:12.767735 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.779004 25629 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:20:12.779031 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:12.780962 25630 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:20:12.751801 31699 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.757393 31934 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.793540 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:12.794201 25638 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:12.770670 31924 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:12.798309 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 131019us
I20260709 12:20:12.783936 31898 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:12.770990 31979 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:20:12.774755 31651 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:20:12.773485 31508 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
W20260709 12:20:12.821519 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 168759us
W20260709 12:20:12.822651 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 453978us
I20260709 12:20:12.799527 31741 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:12.825347 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 56517.6us
I20260709 12:20:12.826378 25752 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:12.785650 31788 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.839433 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:12.846292 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.837740 31741 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.872267 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:12.799156 31934 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.821915 31917 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:20:12.875986 25784 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:12.866305 31898 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:12.860042 31791 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:20:12.883561 31741 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.839944 31821 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.898758 26337 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:12.823261 31784 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:12.894090 31385 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.007s
I20260709 12:20:12.907956 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:12.908442 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:12.912819 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:12.913759 31987 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:20:12.920070 31997 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:20:12.921371 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 49448.2us
I20260709 12:20:12.926201 31734 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:12.938575 31994 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.947767 25707 raft_consensus.cc:2410] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 12:20:12.945955 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 105457us
W20260709 12:20:12.961037 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:12.964087 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 124734us
W20260709 12:20:12.966729 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 158329us
I20260709 12:20:12.969514 25637 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 11: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:12.970882 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.905761 31898 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:12.986905 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.959381 31734 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:12.988816 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.988706 31734 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:12.990996 31734 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:12.992934 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2893) took 1102 ms. Trace:
I20260709 12:20:12.994010 25785 rpcz_store.cc:276] 0709 12:20:11.890462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.003438 (+112976us) service_pool.cc:224] Handling call
0709 12:20:12.160783 (+157345us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5800003398. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:20:12.161071 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:12.713158 (+552087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.992650 (+279492us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b5400099054. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:12.992868 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305443584}
I20260709 12:20:12.930140 31781 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.003921 25751 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:13.005304 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:13.020814 31988 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.155s	user 0.001s	sys 0.001s
I20260709 12:20:13.021924 31821 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.024045 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:12.955209 31850 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.031935 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:13.038935 25638 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:13.071622 31781 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.073282 31898 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.075326 31748 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
W20260709 12:20:13.078629 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 79711.3us
I20260709 12:20:13.079742 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.081296 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.082887 31850 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:13.086831 31993 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:20:13.087656 31993 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:13.088063 31993 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.088382 31993 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:13.093212 31985 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.421s	user 0.004s	sys 0.000s
I20260709 12:20:13.095135 31788 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.100184 31672 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.103790 31992 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:12.969174 31254 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.106300 31254 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:13.031210 31508 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:13.108242 25631 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:20:13.108624 31993 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.103714 25710 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:13.115343 31934 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.117588 31970 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:13.119524 31821 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.120178 31981 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:13.121992 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.123239 25630 raft_consensus.cc:1240] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
W20260709 12:20:13.128374 31989 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 12:20:13.129511 25677 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:13.149175 31831 consensus_meta.cc:287] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.004s
I20260709 12:20:13.138583 31385 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:20:13.143126 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:13.196523 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:13.144814 25637 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:13.149984 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:13.158712 25708 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
W20260709 12:20:13.166635 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.186661 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:13.130379 31982 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:20:13.192636 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 70748.5us
I20260709 12:20:13.200332 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:13.187022 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 57606us
I20260709 12:20:13.192112 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:13.197520 26337 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
W20260709 12:20:13.212057 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 62216.4us
I20260709 12:20:13.213907 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:13.214620 25707 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:20:13.214964 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:13.215816 25708 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:20:13.218899 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58311.5us
I20260709 12:20:13.219081 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:13.200024 25603 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:13.158172 31672 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:13.170336 31981 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:13.236871 25676 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.662s (SENT)
I20260709 12:20:13.238762 25634 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:20:13.222633 31969 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:13.240017 31994 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:13.192778 31421 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:13.237274 31508 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.185957 31821 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.196244 31934 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:13.197623 31788 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.213362 31831 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:13.247247 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48972.6us
I20260709 12:20:13.265954 25637 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:20:13.249339 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:13.255172 25708 raft_consensus.cc:2410] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:20:13.283631 25711 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:20:13.259171 31741 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.291565 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:13.292187 25784 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:13.293197 31741 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.262797 31385 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:20:13.267648 31788 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.257869 31510 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.295585 31510 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:13.269949 31728 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.001s
I20260709 12:20:13.246631 31979 consensus_queue.cc:1059] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:20:13.247380 31995 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.326s	user 0.005s	sys 0.000s
W20260709 12:20:13.297816 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 83363.6us
I20260709 12:20:13.298966 26335 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:13.246305 31981 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:13.296463 31728 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.310443 25751 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.311942 26335 raft_consensus.cc:606] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:13.314041 26335 raft_consensus.cc:614] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:13.300297 31713 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:20:13.314922 25676 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.664s (SENT)
I20260709 12:20:13.312495 31421 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:20:13.246712 31969 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:13.247757 31994 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:13.316789 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:13.242571 32011 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:13.319329 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:13.333428 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 368073us
I20260709 12:20:13.336092 25784 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:13.337643 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:13.322285 31969 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:20:13.360102 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48450.2us
I20260709 12:20:13.350142 25601 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:13.363292 25676 leader_election.cc:336] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.664s (SENT)
I20260709 12:20:13.348871 31748 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:13.349653 31741 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:13.350946 31254 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:13.367762 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:13.372391 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 83273.9us
I20260709 12:20:13.382052 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.382586 32001 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:20:13.383494 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:13.384852 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.375603 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:13.386780 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.686s (SENT) [suppressed 20 similar messages]
I20260709 12:20:13.386436 31994 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.388000 32001 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:13.388898 31898 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:13.390235 31898 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:13.390336 31508 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:13.392252 31784 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:13.350718 31996 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:20:13.392807 31784 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.393909 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.395687 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:13.398801 26335 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:13.403205 31784 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:13.403842 31934 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:13.404522 31934 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.406596 31254 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:13.407318 31781 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:13.407379 31788 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.407904 31254 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.408730 32015 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:13.409627 31728 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.411046 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:13.411669 25707 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:13.413899 31993 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.397138 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.399641 26335 raft_consensus.cc:606] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:13.421773 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:13.424687 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.429143 31993 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.429730 31993 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.404439 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.431536 26338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:13.432243 26338 raft_consensus.cc:606] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:13.432672 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.432928 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.433882 25783 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:13.434458 32012 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:13.434976 32012 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:13.407940 31781 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.435889 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:20:13.435974 31781 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:13.436829 31993 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.436981 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.437755 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:13.438603 31772 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:20:13.441882 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:13.442433 31999 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.575s	user 0.000s	sys 0.002s
I20260709 12:20:13.443368 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.444260 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:13.444877 25630 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.445353 25630 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:13.446184 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.446858 25779 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:13.447340 25707 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:13.447988 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.448477 32015 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:13.449059 32015 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:13.433919 25784 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:13.444398 25781 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:13.450063 32015 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.457269 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.457948 25782 raft_consensus.cc:2376] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17. Re-sending same reply.
I20260709 12:20:13.458177 31630 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:13.458798 32010 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:13.459594 32010 consensus_queue.cc:1243] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer 955233c139344131a847044f17f8d348 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:20:13.459633 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.460427 32010 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:13.460836 32010 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:13.462683 31015 consensus_queue.cc:1059] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:13.463084 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.463521 31015 raft_consensus.cc:3055] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:13.456913 31672 consensus_queue.cc:1059] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:13.466624 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:13.467450 25707 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:13.365744 31510 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.468797 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:13.477478 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:13.480003 26335 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:13.461762 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.437687 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:13.496320 25784 raft_consensus.cc:2376] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11. Re-sending same reply.
I20260709 12:20:13.492455 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:13.463889 31015 raft_consensus.cc:740] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:13.507510 25638 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:13.435370 32012 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:13.517127 31636 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:13.517655 31636 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.517894 32018 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:13.518076 31636 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:13.491739 31993 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:13.530288 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:13.545781 25630 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:13.421087 31969 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:13.542559 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 79562.2us
I20260709 12:20:13.551668 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:13.506067 31900 raft_consensus.cc:3055] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:13.552922 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.555055 31859 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:13.556128 25781 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:13.511106 31510 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:13.562552 25781 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:13.547703 31993 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:13.461745 32010 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.503208 31815 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:20:13.571002 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:13.544615 31728 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.578902 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:13.580004 25710 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 16: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:13.581147 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:13.571866 25785 raft_consensus.cc:2376] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14. Re-sending same reply.
W20260709 12:20:13.598176 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 211428us
I20260709 12:20:13.555784 31900 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:13.693902 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 99435.6us
I20260709 12:20:13.570276 31993 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:13.710834 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 106452us
I20260709 12:20:13.713019 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.718250 25784 raft_consensus.cc:2368] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
W20260709 12:20:13.713898 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 162339us
I20260709 12:20:13.723011 25713 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:20:13.728222 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:13.732775 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.732787 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.734301 25781 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:13.588238 32011 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:13.775875 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46843.9us
I20260709 12:20:13.784139 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.594149 31672 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:13.767360 25676 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.645s (SENT)
I20260709 12:20:13.780815 25710 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:20:13.716358 31998 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.022s	sys 0.096s
I20260709 12:20:13.716730 31983 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:20:13.708247 31508 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:13.589224 31728 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.726658 31741 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.586719 31510 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.809377 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.778389 31734 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:13.791839 31998 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.028s	sys 0.099s
W20260709 12:20:13.814560 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 102062us
I20260709 12:20:13.818158 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:13.776467 32025 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:13.715768 31783 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
W20260709 12:20:13.823258 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 122233us
I20260709 12:20:13.799396 31983 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:13.835930 25780 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:20:13.691148 31015 consensus_queue.cc:260] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.835202 31898 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.846814 31898 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.850643 25710 raft_consensus.cc:1240] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:13.794431 32011 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.853230 31728 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.816817 31741 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.861492 31741 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.862006 25785 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 9.
I20260709 12:20:13.821187 31734 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:13.862723 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 95411.1us
W20260709 12:20:13.893069 25676 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.846s (SENT)
I20260709 12:20:13.863023 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:13.837391 32031 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:13.863027 31015 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:13.864982 31969 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.865741 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:13.866520 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:13.866920 31983 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.831650 31900 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:13.868611 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:13.872078 32021 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 12:20:13.858373 31990 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.938s	user 0.014s	sys 0.017s
I20260709 12:20:13.880662 26335 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:13.857263 31510 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.878893 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:13.857468 31898 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.827430 32010 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:13.820999 32025 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:13.932864 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:13.877408 31728 consensus_queue.cc:1059] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:20:13.807235 31508 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:13.896140 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 74124.9us
I20260709 12:20:13.903947 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:13.885028 31970 consensus_meta.cc:287] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.698s	user 0.016s	sys 0.005s
W20260709 12:20:13.882581 31981 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.005s
I20260709 12:20:13.876204 32000 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:20:13.880168 31850 consensus_queue.cc:1175] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7bf8f20701245bcbaa09cc663025f97" responder_term: 15 status { last_received { term: 15 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 15 index: 10 } }
I20260709 12:20:13.927978 25750 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:13.942222 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:13.912061 31821 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:13.935113 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53801us
I20260709 12:20:13.915799 31975 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:13.874419 31781 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.944164 26335 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.948585 25781 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:20:13.911353 32041 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:13.950580 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.978225 25779 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 18: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:13.950913 31934 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:13.912493 31917 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:20:13.951431 32011 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.957628 31722 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:13.958813 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:13.960067 25787 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:20:13.961407 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 68400.2us
W20260709 12:20:13.986382 25676 leader_election.cc:336] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.853s (SENT)
I20260709 12:20:13.935683 32025 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:13.963258 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2598) took 1403 ms. Trace:
I20260709 12:20:13.947027 31595 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:13.964253 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:13.965708 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1392) took 1067 ms. Trace:
I20260709 12:20:13.943406 31981 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:13.935850 31510 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:13.967434 32008 raft_consensus.cc:3055] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:13.968590 31831 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:13.941545 32000 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:13.963176 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:13.971203 25784 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:13.973982 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.974943 25778 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:20:13.975317 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55557.1us
I20260709 12:20:14.007534 25785 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:13.976183 26335 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:20:13.963766 31734 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:13.986997 25632 rpcz_store.cc:276] 0709 12:20:12.560074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.585866 (+ 25792us) service_pool.cc:224] Handling call
0709 12:20:12.599996 (+ 14130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:12.624704 (+ 24708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:12.624743 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:12.625295 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:20:12.754567 (+129272us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:12.754610 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.754678 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:13.715358 (+960680us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:13.962977 (+247619us) raft_consensus.cc:1685] finished
0709 12:20:13.962997 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:13.963198 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23183,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":97,"spinlock_wait_cycles":60378880,"thread_start_us":128804,"threads_started":1,"wal-append.queue_time_us":247424}
I20260709 12:20:14.003836 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:13.979107 31741 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:13.991161 25635 rpcz_store.cc:276] 0709 12:20:12.898637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.904532 (+  5895us) service_pool.cc:224] Handling call
0709 12:20:12.943054 (+ 38522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:13.217274 (+274220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:13.236731 (+ 19457us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:13.332930 (+ 96199us) log.cc:844] Serialized 26 byte log entry
0709 12:20:13.592858 (+259928us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:13.731242 (+138384us) log.cc:844] Serialized 14 byte log entry
0709 12:20:13.731512 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:13.731606 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:13.965518 (+233912us) raft_consensus.cc:1685] finished
0709 12:20:13.965536 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:13.965645 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62568,"prepare.queue_time_us":184528,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":166053,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":68595,"thread_start_us":436948,"threads_started":2,"wal-append.queue_time_us":557048}
I20260709 12:20:14.011945 25781 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:14.010810 25785 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:13.977571 31636 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.003911 31510 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:14.015940 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.012955 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.001871 32041 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:14.028012 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:14.029284 32011 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:14.030540 32019 log.cc:927] Time spent T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.223s	user 0.003s	sys 0.000s
W20260709 12:20:14.035450 32033 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:20:14.036273 31739 raft_consensus.cc:3055] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:14.036921 31781 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:13.978022 31975 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.038933 31508 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:14.039156 25634 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:14.040241 32000 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:14.043372 32012 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.007s
I20260709 12:20:14.044109 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.008342 31898 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:14.069861 25778 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:14.046687 31981 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.069729 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:14.059367 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 73012.9us
I20260709 12:20:14.071437 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:14.062222 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.065498 25786 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:14.065949 26337 raft_consensus.cc:2368] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13.
I20260709 12:20:14.002086 32008 raft_consensus.cc:740] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:14.033859 31993 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:14.072269 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 288265us
I20260709 12:20:14.061156 31996 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.075248 31994 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.082844 31636 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:14.083390 31994 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.073117 31781 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:14.087165 31781 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.089416 31781 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.089452 25786 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:14.060992 31739 raft_consensus.cc:740] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:14.090937 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:14.091727 31739 consensus_queue.cc:260] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.092245 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.061699 32011 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.094677 31510 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.093228 31636 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:14.095904 32011 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.096052 31636 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:14.097395 31994 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.097445 31636 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:14.104188 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.104835 26341 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:20:14.104707 32041 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.097965 32000 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:14.073447 32012 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.098739 31772 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:14.080559 31898 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:14.099320 25676 leader_election.cc:336] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.927s (SENT)
I20260709 12:20:14.082837 31996 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.099084 31015 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:14.121145 25634 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:14.099550 32008 consensus_queue.cc:260] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.107810 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:14.093247 31739 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:14.130445 31970 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:14.132978 31970 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.134388 31421 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.134963 31421 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.136993 31421 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.096278 31975 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.125280 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.145118 25707 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:14.126956 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:14.147632 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:14.148114 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.148486 31993 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.126662 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.114022 25781 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:14.132001 32008 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:20:14.127707 25751 leader_election.cc:336] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.546s (SENT)
I20260709 12:20:14.139808 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.155208 25708 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:14.131883 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:14.137967 31979 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.142300 31898 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:14.138129 31728 raft_consensus.cc:2764] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:20:14.143978 32012 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.149677 31969 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:20:14.152046 25521 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 60051e5723b2429582407076a003096c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0ae979356d694a09be4b0cd976ea354a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:14.156412 31979 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.132242 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.157825 25630 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:14.158850 31979 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.159101 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:14.159320 32044 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.170605 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:14.135407 31970 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.176147 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:14.174777 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:14.178946 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 64349.5us
I20260709 12:20:14.189649 32000 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.159021 31741 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:14.203033 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.203890 25634 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:20:14.204641 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52455.7us
I20260709 12:20:14.207834 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.208860 25630 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:14.218962 32010 consensus_queue.cc:1059] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:14.219583 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.220229 32010 raft_consensus.cc:3055] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:14.220810 32010 raft_consensus.cc:740] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:14.221148 31983 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:20:14.222836 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 151207us
W20260709 12:20:14.226329 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.220326 25779 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:14.228859 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:14.237879 25601 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.203362 31015 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:14.230544 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:14.240597 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 112913us
I20260709 12:20:14.239933 26338 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:14.246035 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.248406 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.249121 26338 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:14.250125 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.252542 26341 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:14.253974 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.243763 31015 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:14.254978 31015 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:14.255132 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.256059 25637 raft_consensus.cc:2410] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:20:14.256768 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.257782 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.221958 32010 consensus_queue.cc:260] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.258734 31859 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:14.258925 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.259274 31859 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:14.262852 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.263037 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.264056 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.265257 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.266422 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.267452 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.268513 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.269529 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.291266 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.293253 25635 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:14.294154 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.294723 25638 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:14.297147 31015 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.300697 25780 raft_consensus.cc:2410] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:20:14.301172 25633 raft_consensus.cc:2368] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:14.210021 32053 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:14.302306 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.303637 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.239156 32049 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.238647 31981 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:14.312042 25633 raft_consensus.cc:2410] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:20:14.253360 31996 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.326959 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.339931 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.340615 25707 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:14.341651 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:14.342207 25704 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:14.331241 25629 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:20:14.331128 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.344643 25635 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
W20260709 12:20:14.331894 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 103127us
I20260709 12:20:14.344683 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.245299 31998 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:14.343354 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.343014 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.347654 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:14.348542 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:14.349435 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 131241us
I20260709 12:20:14.347864 25705 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:14.336148 26307 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:14.345134 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:14.356479 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.357934 25783 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:14.356411 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.359274 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:14.359341 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.359760 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.360314 25786 raft_consensus.cc:2410] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:20:14.360414 25782 raft_consensus.cc:2393] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 20: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:14.358871 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.363987 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.364320 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.364928 25787 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:14.372702 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.373724 25708 raft_consensus.cc:2393] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:14.375273 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.375433 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.276866 31741 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.376475 25785 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:14.360327 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:14.381137 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 134561us
I20260709 12:20:14.347168 31975 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.384286 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.385818 25676 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:14.390910 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.393359 25638 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:14.395082 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.374601 31996 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:14.359328 31699 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:14.399647 25751 leader_election.cc:336] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.931s (SENT)
I20260709 12:20:14.400130 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.373893 31728 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:14.392022 25781 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:14.402024 25787 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:20:14.376797 31254 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:14.405638 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.407756 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.408461 25633 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:14.400843 32010 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:14.409375 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.409965 25633 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:14.411059 32055 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:14.411549 32055 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:14.411937 32055 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:14.415506 25780 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:20:14.415890 25780 pending_rounds.cc:85] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 7
I20260709 12:20:14.416095 25780 pending_rounds.cc:107] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:20:14.416330 25780 raft_consensus.cc:2889] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:14.416760 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:14.418839 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.419094 31784 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:14.419924 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.407912 25634 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:14.421636 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.439829 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:14.445039 26308 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:14.424010 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.446791 25601 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.347118 31981 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.449244 31981 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.381224 32025 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.420670 31934 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:14.347882 32049 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.452557 32025 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.406275 31728 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.453357 31728 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:14.456463 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50762.5us
I20260709 12:20:14.457485 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.461120 25704 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
W20260709 12:20:14.464113 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.465320 25779 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:14.419956 31784 raft_consensus.cc:993] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:14.467330 25780 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:14.397426 31996 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:14.468197 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 86216.7us
I20260709 12:20:14.473093 31254 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:14.452869 32049 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.492214 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:14.475461 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.405345 31898 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:14.484766 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.451212 31934 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:14.458297 32058 raft_consensus.cc:2764] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:20:14.467351 32011 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:14.496309 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.467728 32044 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.512833 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.518877 25784 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:14.520318 25779 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:20:14.520448 31015 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.520769 25779 pending_rounds.cc:85] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 9
I20260709 12:20:14.521020 25779 pending_rounds.cc:107] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:20:14.521246 25779 raft_consensus.cc:2889] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:14.474049 32012 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:14.491757 31996 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:14.421166 32006 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 12:20:14.438287 31859 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:14.538686 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 94792us
I20260709 12:20:14.540010 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:14.540570 31859 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.541455 25633 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:14.542066 31630 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.544085 32054 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:14.406142 31975 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.545645 25785 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:14.528568 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.487661 31917 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:14.535001 32057 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.535468 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.488417 31722 raft_consensus.cc:1081] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:14.500990 31781 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:14.548966 25787 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:14.550886 31015 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:14.555351 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:14.555966 32057 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.556711 32053 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:14.556963 25778 raft_consensus.cc:2393] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:14.542155 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:14.593434 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64966.4us
I20260709 12:20:14.595005 25601 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.596047 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.577262 25704 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:14.566110 32039 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:14.607189 25603 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:14.605544 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 63462.9us
I20260709 12:20:14.605736 25781 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:14.608240 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.571739 31917 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:14.611506 25676 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:14.634630 31015 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.624271 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:14.628435 32037 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:20:14.614391 31421 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.612758 32044 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:14.631979 32018 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.642441 31630 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:14.660840 31975 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.662874 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.664109 31015 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:14.664633 31015 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:14.666395 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:14.659937 25781 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:14.612057 31781 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:14.626835 31699 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.671293 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.670099 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.622658 31859 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:14.696558 25631 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:14.688000 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.697223 25778 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:14.697505 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:14.698113 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:14.699043 26341 raft_consensus.cc:2410] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:20:14.652110 32057 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:14.706336 26343 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:14.613013 31772 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:20:14.609149 31636 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.652318 32058 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:20:14.658806 32044 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:14.622953 32039 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:14.736217 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.737648 25778 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:14.711025 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:14.738603 26337 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:14.739800 25708 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:20:14.714335 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.671808 31672 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:14.717535 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.718384 31917 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.724844 31772 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:14.725929 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:14.750157 25631 raft_consensus.cc:2410] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:20:14.699987 31421 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:14.748895 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.703439 31015 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.753803 26335 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:14.753052 26344 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:14.672367 32025 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:20:14.652750 31984 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 12:20:14.755770 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 89442.7us
I20260709 12:20:14.672113 32054 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:20:14.756057 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.759419 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.760735 26341 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:14.762639 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.715585 31859 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:14.738746 32044 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.763468 25631 raft_consensus.cc:2376] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 18. Re-sending same reply.
I20260709 12:20:14.762743 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:14.797104 26343 raft_consensus.cc:2393] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:14.715636 32058 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:14.714339 31996 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.808197 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.758584 31741 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.751503 31772 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:14.772459 31900 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:14.778110 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.755280 31672 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:14.780002 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:14.783728 31739 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:14.785852 31859 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.787503 32011 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.788522 31898 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.790361 32039 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:14.751287 32053 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:14.809327 32058 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:14.767796 31636 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:14.815662 25633 raft_consensus.cc:2376] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 955233c139344131a847044f17f8d348 in term 13. Re-sending same reply.
I20260709 12:20:14.820236 25782 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:14.807739 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:14.761411 31994 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:20:14.785931 31934 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.837947 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:14.751137 31728 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.843895 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:14.845402 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:14.831825 31699 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:14.849308 32008 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.679s	user 0.012s	sys 0.000s
W20260709 12:20:14.900400 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52583.8us
I20260709 12:20:14.901146 25786 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:14.901894 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:14.823783 31898 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:14.906458 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54925.8us
I20260709 12:20:14.846688 31934 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:14.843503 32025 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:14.916327 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.917069 25633 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:14.917315 25601 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.917974 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:14.918553 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:14.899397 31975 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:20:14.922755 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 166807us
I20260709 12:20:14.925791 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.840140 32055 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.929767 25633 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:14.903342 32057 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:14.935050 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.937340 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.938053 25708 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:14.942652 31788 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:20:14.922410 31979 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.916190 32025 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.955689 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:14.956533 25704 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:14.958595 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.959964 25787 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:14.934340 32008 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:14.962368 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:14.912467 31970 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:20:14.961015 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:14.970927 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.846125 31772 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.935446 32057 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.939647 31975 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:14.999291 31975 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:14.947062 31739 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:15.002957 31739 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:15.003461 31739 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.005488 31739 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:14.980932 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:14.911230 31672 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.971473 25787 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:14.954018 32037 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:20:14.963280 32025 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:14.968940 25784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:15.058528 25784 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:15.059089 25784 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:14.990422 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:14.991626 31996 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:14.953989 31791 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:20:14.963464 31979 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:14.993103 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:14.961496 32074 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:20:14.995344 25778 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:20:14.969043 31699 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:20:14.964668 32042 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:20:15.016698 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 56491.6us
I20260709 12:20:14.969309 31741 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:15.022215 26337 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:15.026919 26338 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:15.028951 25708 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:20:15.007189 32057 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:15.057147 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 138676us
I20260709 12:20:15.071883 32074 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:15.080683 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.025413 32012 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:15.069069 32037 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:15.109881 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:15.110759 25708 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:15.082521 31996 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:15.134577 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.086238 32042 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:15.084354 32074 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:15.138780 25784 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.145651 25637 raft_consensus.cc:2410] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 12:20:15.151037 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 77864.8us
I20260709 12:20:15.154399 26337 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:20:15.159803 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 169456us
I20260709 12:20:15.160583 25708 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:15.111656 32012 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:15.129612 31739 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:15.178509 25784 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:15.184172 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:15.182255 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 96231.8us
I20260709 12:20:15.186025 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.200721 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:15.202840 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 68390us
I20260709 12:20:15.203917 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.208752 32091 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:15.209357 32091 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.212713 32091 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:15.214820 26337 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:15.215301 26337 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:15.217732 25778 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:15.224931 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:15.192765 31981 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.216291 26337 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.167546 31741 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:20:15.237949 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:15.180877 31900 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:20:15.243948 26337 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:15.198089 32076 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:15.247365 25708 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:15.258597 26337 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:15.205770 32084 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:15.263373 32008 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:15.228536 32074 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.195713 31734 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:20:15.197813 32053 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:20:15.229003 32010 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:20:15.235893 32055 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:15.239368 31850 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:15.236877 31898 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:15.275403 32073 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 12:20:15.295907 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:15.286263 32074 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:15.301390 32076 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:15.301546 31979 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.216064 32037 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.301959 32089 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:20:15.302785 32072 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 12:20:15.303079 32086 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:20:15.248951 32042 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.257045 32091 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:20:15.244760 31981 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.260444 31739 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:15.290038 31741 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:15.390324 25676 leader_election.cc:400] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13.
I20260709 12:20:15.363714 32049 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.429723 25676 leader_election.cc:403] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:20:15.374269 32080 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:20:15.359673 31859 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.265125 32012 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:15.438884 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 65039us
W20260709 12:20:15.435851 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 249973us
I20260709 12:20:15.367758 32089 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.441437 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.422252 32084 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:15.438102 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 207939us
I20260709 12:20:15.375548 31979 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:15.435021 32077 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:20:15.362309 32076 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.446022 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:15.450788 32097 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:20:15.464819 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.447978 31784 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:15.466346 31784 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.466830 31784 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.468957 31784 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:15.469311 32071 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.458s	user 0.001s	sys 0.001s
I20260709 12:20:15.469731 32099 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:15.437448 32049 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.473371 32049 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:15.475395 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.439127 32055 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13.
W20260709 12:20:15.507462 32087 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.264s	user 0.003s	sys 0.000s
I20260709 12:20:15.440251 31859 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.442992 31917 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:15.509061 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59651.1us
I20260709 12:20:15.430858 31741 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:15.449110 31421 consensus_meta.cc:287] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.006s
I20260709 12:20:15.527637 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:15.528445 25711 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:15.443367 32089 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:15.542778 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 66243.6us
W20260709 12:20:15.577200 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.583032 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:15.515655 31934 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.003s
I20260709 12:20:15.530737 32092 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.447428 31979 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.591756 25713 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:20:15.278909 31672 consensus_queue.cc:1175] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "10d0c7349085463595aedaf7c071e742" responder_term: 14 status { last_received { term: 12 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:20:15.476337 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:15.556363 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 318465us
I20260709 12:20:15.558184 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.447144 32084 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.527383 32098 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:15.550719 32008 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.547420 31975 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:15.607884 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:15.612038 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 53948.4us
W20260709 12:20:15.616782 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 140767us
W20260709 12:20:15.621485 32094 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:20:15.623505 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:15.624517 32084 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.582511 32100 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:20:15.581035 31421 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:15.626698 32008 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.629019 32008 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.624800 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:15.627106 31979 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:15.581153 31741 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:15.630606 25706 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:20:15.600685 32096 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:15.632763 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.632961 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.639014 25785 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:15.620092 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.636811 25778 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:15.542196 31739 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.615935 31996 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:15.646468 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:15.646709 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:15.647243 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:15.648342 25635 raft_consensus.cc:2393] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:15.650576 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:15.617803 31734 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:15.652180 25630 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:15.585933 31917 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.655066 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:15.657584 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:15.657878 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.678964 25785 raft_consensus.cc:2368] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:20:15.680008 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.680663 25785 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:15.659439 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:15.682866 25634 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:15.660559 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:15.654028 32044 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.863s	user 0.006s	sys 0.007s
I20260709 12:20:15.646202 32098 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:15.661007 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.688998 32012 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:15.689987 25778 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:20:15.689854 32012 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.662703 25787 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:20:15.693905 32012 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:15.663512 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:15.665057 32101 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:15.694885 25711 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:15.695050 32101 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:15.697408 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46980.4us
I20260709 12:20:15.697665 32101 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:15.655792 32099 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.698427 32099 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.699808 25782 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 18: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:15.665520 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.700881 25786 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:15.700950 32099 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:15.667292 31859 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.701884 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.670679 32095 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.664480 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.645486 32039 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:15.702668 32095 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.702939 25783 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:15.702560 25786 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:15.705302 32095 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:15.702051 31859 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:15.707021 31859 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.702973 32039 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.707459 31859 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.709157 32039 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:15.685357 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.714442 31998 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:15.664608 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44586.3us
I20260709 12:20:15.715804 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:15.656133 32096 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:15.635058 31975 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.613000 32092 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.661566 31981 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.740120 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:15.740883 26344 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:15.741315 31975 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.743831 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.722858 32095 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:15.749589 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:15.756143 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.758813 25676 leader_election.cc:400] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16.
I20260709 12:20:15.760673 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:15.763129 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:15.760169 25783 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:15.765199 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.775058 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:15.718254 32044 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.770606 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:15.786561 25630 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:15.771056 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:15.772852 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:15.789161 25634 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:15.791368 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:15.759686 32095 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.767408 31421 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:15.802836 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:15.803025 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:15.803761 26340 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:15.804901 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:15.807021 25636 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:15.773876 32076 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:15.805609 26336 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:15.801985 32095 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:15.773278 32098 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:20:15.786474 31734 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:15.795925 31784 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:15.795707 32039 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 12:20:15.819648 32058 consensus_meta.cc:287] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.983s	user 0.011s	sys 0.001s
W20260709 12:20:15.821385 32057 consensus_meta.cc:287] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.001s
I20260709 12:20:15.822853 32092 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.770514 32103 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:15.824761 25783 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:20:15.825943 26344 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:15.804322 31421 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.871411 25638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1412) took 1458 ms. Trace:
I20260709 12:20:15.877570 25638 rpcz_store.cc:276] 0709 12:20:14.413350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.815206 (+401856us) service_pool.cc:224] Handling call
0709 12:20:15.836063 (+1020857us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b5800000c98. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286aa22 00007f29bb7838d9 00007f29ba8ddee0 00007f29ba9a70ab 00007f29ba9a7131 00007f29ba968fc0 00007f29ba953929 00007f29ba9533d2 00007f29b3ecabd4
0709 12:20:15.869171 (+ 33108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.871097 (+  1926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.871123 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:15.871165 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:15.871200 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.871254 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:15.871357 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3815168}
I20260709 12:20:15.799300 32055 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:20:15.903143 26336 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:15.891127 26341 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:15.883637 31421 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:15.914479 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:15.892357 32109 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:20:15.875054 32058 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.919312 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:15.921581 26341 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:15.891598 32090 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:15.924926 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 159607us
I20260709 12:20:15.877656 32057 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:15.931367 25710 raft_consensus.cc:1240] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 9. Current term is 10. Ops: []
I20260709 12:20:15.923339 32103 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:15.933468 31917 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:15.937273 32044 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:15.968709 25707 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:15.943382 31421 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.923316 31784 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.950963 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:15.954746 31595 consensus_queue.cc:1059] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:15.959458 31969 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:20:15.961517 32079 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.768s	user 0.000s	sys 0.002s
I20260709 12:20:15.964411 31739 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:15.965144 31970 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:15.928153 32090 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.973184 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:16.037955 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 64845.2us
I20260709 12:20:15.917695 31859 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:15.966161 32089 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:15.979553 32025 consensus_meta.cc:287] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.906s	user 0.010s	sys 0.002s
I20260709 12:20:15.948832 32103 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:15.987995 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:15.988406 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:15.992010 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.043602 25630 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:15.993727 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.001150 31728 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.002249 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:16.047428 26341 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:16.008606 32058 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.014678 32057 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:16.022204 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.044523 25638 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:16.054610 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.040262 32090 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.042299 31979 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:16.051916 31859 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.035823 31970 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:16.062975 31970 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.064227 25637 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:16.052131 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.065171 25630 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:16.066051 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.067245 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.032222 31969 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:16.068606 31969 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.069833 31969 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:16.060660 31979 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.070864 31979 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.071362 26342 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:16.059013 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.072228 25638 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:20:16.073020 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.073278 32099 consensus_queue.cc:1059] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:16.073576 25638 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:20:16.075235 25638 raft_consensus.cc:1240] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:16.075774 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:16.076759 31636 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:16.077150 31636 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.077549 31636 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:20:16.080017 32074 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.000s
I20260709 12:20:16.046685 32025 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.083753 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:16.084056 32025 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.031252 31784 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.084864 31722 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:16.085350 31722 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.085673 31722 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:16.087044 31784 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.090895 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:16.092068 25632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2632) took 1738 ms. Trace:
I20260709 12:20:16.092499 25632 rpcz_store.cc:276] 0709 12:20:14.353664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.453651 (+ 99987us) service_pool.cc:224] Handling call
0709 12:20:14.454053 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:14.454279 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:20:14.775463 (+321184us) log.cc:844] Serialized 14 byte log entry
0709 12:20:15.960948 (+1185485us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:16.091383 (+130435us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b180026e540. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f23a
0709 12:20:16.091675 (+   292us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:16.091729 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:16.091776 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.091868 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.092000 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":393483,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3072,"raft.queue_time_us":1793,"spinlock_wait_cycles":19860224,"thread_start_us":478379,"threads_started":2,"wal-append.queue_time_us":365389}
I20260709 12:20:16.094317 32103 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.095098 32010 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:16.095547 32010 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:16.096474 32010 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.097609 32010 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:16.099016 25630 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:20:16.032977 32118 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:16.131604 25629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2635) took 1772 ms. Trace:
I20260709 12:20:16.135483 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:16.037837 31739 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.138056 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.141471 25709 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:16.138773 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:16.142421 25708 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:16.137522 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.143286 25710 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:16.143216 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:16.132151 25629 rpcz_store.cc:276] 0709 12:20:14.359057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.527373 (+168316us) service_pool.cc:224] Handling call
0709 12:20:14.553255 (+ 25882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:14.570320 (+ 17065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:14.570352 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:14.589965 (+ 19613us) log.cc:844] Serialized 26 byte log entry
0709 12:20:14.810077 (+220112us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:14.810116 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.468600 (+658484us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:16.131187 (+662587us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b54003007d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:16.131268 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:16.131330 (+    62us) raft_consensus.cc:1685] finished
0709 12:20:16.131350 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.131535 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":174,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4845,"spinlock_wait_cycles":91063936,"thread_start_us":177440,"threads_started":1,"wal-append.queue_time_us":312811}
I20260709 12:20:16.097632 32089 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.149175 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:16.118031 31831 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:16.155107 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.043150 31741 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.157188 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:16.162879 26341 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
W20260709 12:20:16.167241 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 128293us
I20260709 12:20:16.150218 31734 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.092499 31636 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.175760 25785 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:20:16.155611 32099 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:16.171226 25708 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:16.152045 32076 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:16.154209 32057 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:16.164279 32101 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:16.165558 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:16.169555 31996 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:20:16.175324 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 136673us
I20260709 12:20:16.189221 31996 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:16.130642 31722 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.180968 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:16.147567 32098 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:20:16.193535 25752 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:16.160769 31979 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.160929 32090 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.185227 32101 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:16.196242 31996 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:16.196169 32093 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:16.187152 32057 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.177713 31734 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.235790 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3310) took 1384 ms. Trace:
I20260709 12:20:16.243175 25704 rpcz_store.cc:276] 0709 12:20:14.851209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.851426 (+   217us) service_pool.cc:224] Handling call
0709 12:20:14.851859 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:14.958435 (+106576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.235533 (+1277098us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400105254. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:16.235720 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240187776}
W20260709 12:20:16.242300 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61404.5us
I20260709 12:20:16.251085 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.252451 25630 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:16.252942 25629 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:16.250381 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:16.253592 25704 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.256127 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.257795 32130 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:16.258392 32130 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.246551 32055 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:16.266173 32055 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.266642 32055 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:16.227223 31898 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:16.267988 31898 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:16.268981 31898 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.239391 32101 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:16.214699 31728 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:16.281303 31728 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:16.222921 32093 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:16.218473 31741 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.238773 32090 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.232765 31421 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.343312 31734 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.231870 31739 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.206406 31722 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.230556 31636 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.281231 32103 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:16.345858 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1428) took 1162 ms. Trace:
I20260709 12:20:16.346297 26337 rpcz_store.cc:276] 0709 12:20:15.183772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.183916 (+   144us) service_pool.cc:224] Handling call
0709 12:20:16.345666 (+1161750us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b5400471654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:16.345793 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67915136}
I20260709 12:20:16.346566 31996 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.348080 31831 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:16.348688 31831 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:16.280046 32095 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.237056 32057 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.352361 31996 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.352396 32095 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.299316 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:16.358774 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59559.7us
I20260709 12:20:16.359396 32091 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:16.359680 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:16.360929 31739 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:16.361455 31739 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:16.362509 31739 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.367129 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.367844 25635 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:16.304004 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:16.310173 32093 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:16.282544 31728 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.338231 32099 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.377230 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:16.378248 26338 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:16.340101 31979 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.379724 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:16.386090 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:16.386752 25782 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.386792 25787 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:20:16.388223 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.383205 25704 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
W20260709 12:20:16.390107 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.391884 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.393690 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.394394 25634 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:16.403698 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.404628 25637 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.405128 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:16.406177 25678 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.409402 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.407258 32130 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.392982 31781 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.384028 32091 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.428650 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.429334 25631 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:16.416234 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.418208 25784 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:16.431731 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.433907 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:16.434563 26338 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.403630 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:16.421181 25708 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:16.436049 25634 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:20:16.429435 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:16.438311 25677 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:16.438911 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.440277 31788 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:16.443317 26308 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:16.440816 31788 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.384155 31722 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:20:16.423826 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.470075 31969 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:16.427327 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.446662 25635 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
W20260709 12:20:16.449512 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46566.5us
I20260709 12:20:16.420998 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:16.449895 26337 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:20:16.450793 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.451594 32100 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:20:16.451707 31421 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:16.453928 32018 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:16.454644 31859 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:16.455715 31900 consensus_queue.cc:1059] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:20:16.457028 32044 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:16.514038 32044 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:16.514453 32044 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:16.458688 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.494879 25782 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
W20260709 12:20:16.516925 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:16.490733 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 47503.9us
I20260709 12:20:16.517750 31979 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:16.518466 31979 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.446293 32091 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.521256 31979 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.524716 26308 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:16.495925 25630 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.498768 25713 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:16.526041 32118 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:16.446900 32090 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:16.503203 31421 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.528093 31421 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:16.511458 32100 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:16.531102 31900 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:20:16.531525 31900 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:16.531955 32100 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.532496 31900 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.533478 31917 raft_consensus.cc:2804] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:16.533806 31900 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:16.534281 31917 raft_consensus.cc:697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:16.441794 31781 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.535636 31917 consensus_queue.cc:237] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.537009 31781 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.539705 32010 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:16.499183 26337 pending_rounds.cc:85] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 13
I20260709 12:20:16.498097 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:16.503588 25705 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:16.540819 25710 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:16.541055 26337 pending_rounds.cc:107] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:20:16.504657 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.391950 32099 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.512887 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:16.546173 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 3034) took 1025 ms. Trace:
I20260709 12:20:16.492174 25704 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:16.541379 26337 raft_consensus.cc:2889] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:16.478048 31788 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.548656 25705 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:16.546607 25706 rpcz_store.cc:276] 0709 12:20:15.520515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.642288 (+121773us) service_pool.cc:224] Handling call
0709 12:20:15.642608 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.736677 (+ 94069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.736711 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:15.736764 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:15.736799 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.545962 (+809163us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b5400203d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:16.546008 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.546119 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215811968}
I20260709 12:20:16.541721 25708 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
W20260709 12:20:16.542243 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:16.552816 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.544164 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:16.557166 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:16.558293 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:16.558293 26338 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:16.524917 32101 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.563078 25780 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:16.578454 25704 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:16.575241 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:16.617810 25705 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.620015 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:16.620966 25704 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:16.549659 31781 raft_consensus.cc:2764] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:20:16.619956 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.556633 31859 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:16.624281 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.625963 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:16.639050 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:16.645417 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 84728.7us
I20260709 12:20:16.648257 25601 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:16.657053 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 91533.2us
I20260709 12:20:16.565282 32089 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:16.548240 32010 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.563493 32012 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.108s
I20260709 12:20:16.687394 25713 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:20:16.699046 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:16.703308 25603 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:16.671386 31979 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:16.673269 31898 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:16.700171 25713 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:16.672223 31859 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.654161 32055 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.693176 32090 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:16.693755 32138 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:16.703543 32092 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:16.726645 31969 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.704731 32089 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.693825 32140 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:16.728788 32041 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.755517 25705 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
I20260709 12:20:16.693848 32139 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 12:20:16.743793 31979 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.732391 32095 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:20:16.753046 32018 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:16.757927 32138 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:16.769915 32136 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:16.757062 31969 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.759037 32089 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.764017 32041 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.777130 31975 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:16.765893 32140 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.794378 32118 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:16.822809 32118 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.824505 32140 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.824931 32118 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.825124 32140 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.825613 32140 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.827171 32012 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:16.827584 32012 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:16.828464 32012 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.829741 32012 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.832245 31979 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.833173 31831 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.799827 32136 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.838387 32136 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.783399 32093 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.841167 32093 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.843478 32041 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.775702 32092 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.850749 25637 raft_consensus.cc:2410] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:20:16.866652 32138 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:16.867341 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 674816us
I20260709 12:20:16.868506 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:16.855499 32133 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:20:16.872505 31970 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:16.844926 32090 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.874801 31595 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:16.876830 31969 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.879442 32092 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.880199 32010 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.849069 31975 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.882200 31831 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.882949 31975 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.883587 31421 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.870020 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:16.872535 32089 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.897481 31969 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:16.898212 31969 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:16.898401 25707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50646 (request call id 1076) took 1745 ms. Trace:
I20260709 12:20:16.901662 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2444) took 1185 ms. Trace:
I20260709 12:20:16.917980 25785 rpcz_store.cc:276] 0709 12:20:15.716803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.724922 (+  8119us) service_pool.cc:224] Handling call
0709 12:20:15.725408 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.176311 (+450903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.901412 (+725101us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540035a2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:16.901584 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035022592}
I20260709 12:20:16.902019 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:16.916613 25707 rpcz_store.cc:276] 0709 12:20:15.153525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.153690 (+   165us) service_pool.cc:224] Handling call
0709 12:20:15.153986 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.969216 (+815230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.898084 (+928868us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b5400105754. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:16.898322 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873314048}
I20260709 12:20:16.903403 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:16.848896 32099 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.849323 32101 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.890560 31981 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.892791 31772 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.929962 31981 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.930891 31859 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.931701 26338 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:20:16.906070 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:16.911113 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.932981 26335 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.909083 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.904862 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.886919 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2637) took 2506 ms. Trace:
I20260709 12:20:16.936683 25636 rpcz_store.cc:276] 0709 12:20:14.380642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.670377 (+289735us) service_pool.cc:224] Handling call
0709 12:20:14.694865 (+ 24488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.807577 (+1112712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.886632 (+1079055us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002abcd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:16.886829 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2557175168}
I20260709 12:20:16.914811 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.907841 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.920682 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:16.921574 26341 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.908422 32055 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.927946 32140 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:16.928606 32091 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:16.905946 25637 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:20:16.945534 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.947257 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:16.950886 25631 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:16.951395 31015 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.953058 31015 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:16.954324 25637 pending_rounds.cc:85] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 11
I20260709 12:20:16.954746 25637 pending_rounds.cc:107] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:20:16.955096 25637 raft_consensus.cc:2889] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:16.965334 26335 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:16.969628 25636 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:16.970031 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:16.970517 26341 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:16.970840 25632 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:16.915092 32074 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:16.994589 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.934617 31722 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.980275 25676 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:16.940299 31784 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.007062 31015 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:16.957506 32130 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:17.013242 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.029184 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:17.032014 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.906548 31788 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:17.038730 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 307168us
I20260709 12:20:16.934370 32038 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
W20260709 12:20:17.045883 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.046895 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.997568 31969 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:16.911664 32044 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:16.966861 32092 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:20:17.027657 32008 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:17.052657 31015 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:17.059060 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:16.997072 31595 consensus_queue.cc:1059] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:20:17.064643 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:17.035025 32130 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:17.069024 26336 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:20:17.028421 31722 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.076715 26337 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:20:17.074388 32130 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:17.042071 31734 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:20:17.035032 31784 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.045184 32074 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.052925 32076 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:20:17.089377 25750 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:17.074834 31975 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:20:17.095521 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 89853.6us
W20260709 12:20:17.093662 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 417924us
I20260709 12:20:17.102694 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:20:17.094030 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 421955us
I20260709 12:20:17.094336 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.081138 32144 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:17.062016 31969 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.113250 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.113339 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.114192 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:17.115602 25785 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:20:17.115795 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.116456 25708 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:17.117624 26337 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:20:17.114259 25787 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:17.119339 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:17.118127 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.121310 25713 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
W20260709 12:20:17.110184 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.122394 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.122706 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:17.123349 25778 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:17.128450 25778 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:17.131258 26340 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:20:17.084460 32149 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:17.136205 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:17.136915 25638 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:17.136308 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.138978 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.093961 32044 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:17.152194 25787 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:17.158993 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.159786 25713 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:17.165400 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:17.108108 32091 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:17.170974 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.145309 25603 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.170431 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:17.097705 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:17.173583 25634 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:17.092880 31975 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:17.176833 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 79226.7us
I20260709 12:20:17.177721 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.177568 31975 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.178548 25712 raft_consensus.cc:2410] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:20:17.178784 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:17.179881 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.180217 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:17.180675 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:17.180608 32055 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:17.181245 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:17.181964 25710 raft_consensus.cc:2368] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:20:17.182456 32055 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:17.180858 25708 raft_consensus.cc:2368] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:17.183270 32055 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:17.183825 32055 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:17.185663 32092 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:17.186241 32092 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:17.182255 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:17.134295 32105 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.175810 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:17.193559 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48345.7us
I20260709 12:20:17.169059 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:17.127633 32144 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:17.194293 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.194466 25637 raft_consensus.cc:2393] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:17.194938 25786 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:17.133424 32018 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:17.191891 25632 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:17.135459 31898 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:17.176383 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.200527 25780 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:17.193902 25638 raft_consensus.cc:2410] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:20:17.174891 32008 raft_consensus.cc:2764] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:20:17.195603 25603 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.135828 31421 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:17.198081 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:17.119488 32012 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:20:17.119341 32098 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:20:17.171399 32091 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.206452 32138 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:17.178805 31975 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:17.261417 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:17.201828 32018 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:17.279750 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.188090 31934 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:20:17.211952 31630 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:17.221479 32091 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:17.220448 32138 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.184850 32055 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.211006 32099 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:17.200201 32105 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.323693 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.323949 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.323340 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.351034 25786 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:17.279983 25632 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:17.329010 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.352732 25784 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:17.210358 32147 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.125s
I20260709 12:20:17.284760 32018 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:17.281201 31739 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:17.351682 25778 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:17.296267 31898 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:17.322530 32138 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:17.392992 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 197451us
I20260709 12:20:17.393656 31994 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:20:17.394414 32089 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:17.403326 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.414867 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.415443 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.415926 25784 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:17.416497 25787 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:17.416790 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:17.417011 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 202079us
I20260709 12:20:17.419279 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:17.418622 25784 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:20:17.420040 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 237834us
I20260709 12:20:17.421324 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.421974 25705 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 11.
I20260709 12:20:17.412667 25677 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.423169 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.424053 25705 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:17.424770 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.425493 25708 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:17.426476 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:17.426738 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.429332 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.429548 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:17.430004 25636 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:17.432482 25706 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:20:17.387513 31788 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.436601 25638 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:20:17.438071 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.444355 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.435670 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:17.452972 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:17.453572 25634 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:17.453856 25601 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:17.457471 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.458166 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.458793 25786 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:17.458928 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.459549 25780 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:17.414930 32089 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.411059 32074 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.414832 32100 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:17.380331 32099 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.438308 32103 raft_consensus.cc:2804] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:17.321713 32092 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.399686 31739 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.505430 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:17.515823 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 80682.8us
I20260709 12:20:17.523973 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:17.524631 25705 raft_consensus.cc:3060] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:20:17.460930 31722 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.425643 31421 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:20:17.475063 32089 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:17.537870 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.322245 32144 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.608604 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:17.511018 31969 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:17.511322 31784 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:17.619334 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 86354.2us
I20260709 12:20:17.528013 32149 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:17.521163 31859 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:17.622862 25705 raft_consensus.cc:2468] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:17.629063 25786 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:17.632462 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:17.633795 25705 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:17.510252 32100 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.644696 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.618230 32103 raft_consensus.cc:697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:17.649170 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.628775 32173 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:17.625946 31781 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:20:17.682411 32105 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.701727 32149 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:17.702135 32091 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.671428 25708 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:20:17.673465 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:17.678735 32157 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:20:17.679706 31981 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.681773 32099 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:17.615520 31898 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.683704 32103 consensus_queue.cc:237] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.664915 32173 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:17.715131 32044 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.006s
I20260709 12:20:17.729239 32091 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:17.685591 31772 consensus_meta.cc:287] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.002s
I20260709 12:20:17.685911 31739 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:17.686093 32169 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 12:20:17.755592 32130 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.002s
W20260709 12:20:17.705952 32164 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.306s	user 0.001s	sys 0.006s
I20260709 12:20:17.756420 32130 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.692093 32100 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:17.634195 31784 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 12:20:17.763445 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 309662us
W20260709 12:20:17.710942 32158 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.361s	user 0.000s	sys 0.010s
W20260709 12:20:17.708451 32161 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.355s	user 0.001s	sys 0.008s
W20260709 12:20:17.707222 32156 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
W20260709 12:20:17.709661 32155 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
I20260709 12:20:17.733259 32173 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:17.824116 32167 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:20:17.702363 32149 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:17.669745 32175 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:17.826877 32175 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:17.743110 25630 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:17.744486 25708 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:20:17.827872 31015 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.703809 25636 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:17.828284 31015 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:17.763401 31975 consensus_meta.cc:287] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 12:20:17.805578 25601 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:17.806946 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 409998us
W20260709 12:20:17.807636 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 182611us
I20260709 12:20:17.836163 25636 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:17.837316 25601 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:17.838106 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.840809 25677 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.822561 32100 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.841514 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:17.850287 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:17.852602 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:17.857882 31970 raft_consensus.cc:3055] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:17.858290 31970 raft_consensus.cc:740] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:17.858812 31630 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:17.859166 31970 consensus_queue.cc:260] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.842969 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:17.847077 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.809257 32138 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.858641 32100 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.867929 32100 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.869112 32138 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.870702 32096 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:20:17.872094 32160 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.587s	user 0.000s	sys 0.001s
I20260709 12:20:17.876727 32176 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:17.850021 32181 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:17.839499 32130 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:17.879249 32176 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:17.880148 32176 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:17.881429 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.832194 32089 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:17.884040 32089 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:17.885121 32171 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:20:17.875483 25708 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 18: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:17.875196 32096 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:17.861268 32018 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.886988 32096 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:17.887984 32096 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:17.888098 32018 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:17.876592 32170 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:20:17.872978 31630 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:17.902843 31630 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:17.907523 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 55836.2us
I20260709 12:20:17.908656 31917 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:20:17.874904 31970 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:17.925648 32175 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:17.829216 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 380712us
I20260709 12:20:17.932132 25636 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:17.932984 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:17.912153 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 48555.9us
I20260709 12:20:17.938083 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.941303 25782 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:17.941764 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:17.942560 25713 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:17.939997 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:17.945401 25708 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:17.946324 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.949079 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:17.950161 25782 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:17.953056 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.953691 25782 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:17.953912 25708 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
W20260709 12:20:17.956017 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:17.963256 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 54832.2us
W20260709 12:20:17.963388 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 54921.1us
I20260709 12:20:17.987557 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:17.963519 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:17.971346 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:17.944319 32095 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:17.985937 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:17.980717 25713 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:17.984037 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:17.994531 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.970083 32175 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:17.978989 25782 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
I20260709 12:20:17.999791 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:18.009091 25601 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:18.006109 31831 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:18.004366 31784 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.002979 32055 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:17.978077 31934 consensus_queue.cc:1059] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 12:20:18.019768 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.011380 32181 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:18.019589 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:18.023770 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:18.015172 32168 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:18.006470 32095 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.029168 32180 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.029717 32184 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:18.034703 31784 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:18.024933 31722 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.015476 31772 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:18.064112 31772 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.067214 31772 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.075850 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:18.080219 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.080901 31015 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.081465 31015 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:18.081909 31015 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.082520 25630 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:20:18.084444 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:18.085045 25633 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:18.085503 31781 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:18.086094 31781 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.088289 31781 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.088660 32095 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.089020 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:18.089608 25707 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:18.085112 31831 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.091741 31831 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.093988 31831 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.093734 32149 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:18.039885 32165 log.cc:927] Time spent T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.516s	user 0.003s	sys 0.000s
I20260709 12:20:18.022994 32181 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.008037 32176 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.016s
W20260709 12:20:18.063239 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.111514 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:18.131539 25636 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:18.111826 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:18.098307 25603 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:18.123045 25633 raft_consensus.cc:1240] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:18.132481 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:18.048624 32173 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.027951 31934 consensus_queue.cc:1243] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:18.024885 32055 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:18.125911 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:18.127537 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:17.625375 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:18.141459 25704 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:18.141703 25632 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:18.153823 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:18.155540 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:18.157274 25786 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:18.157752 25782 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:18.147010 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:18.158883 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:18.140137 32089 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:18.130657 32180 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.130586 32181 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.168864 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:18.173379 26340 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:18.170791 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:18.174362 25704 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 10.
I20260709 12:20:18.168632 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:18.172310 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:18.156589 32184 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:18.164156 32100 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:18.175705 25780 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:18.183115 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:18.130252 31981 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:20:18.190701 25603 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:18.169993 32180 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.175767 31831 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:18.170706 32089 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:18.061566 32168 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.188771 32012 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.202914 32197 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:18.203402 32197 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.203737 32197 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:18.167539 32018 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:20:18.199092 32012 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.191676 32100 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:18.195560 31672 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:20:18.197881 31831 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:18.192345 31722 consensus_queue.cc:1059] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:20:18.242856 32184 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.247269 31831 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:18.248479 32188 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:20:18.251055 31722 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:18.251631 31722 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:18.252827 31722 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.216559 32055 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.262358 32173 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.262084 32089 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.214603 32012 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.267565 25710 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:18.268078 32149 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.218255 32100 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:18.231730 31672 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.274698 31672 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:18.278285 32163 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:18.278877 32163 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:18.292584 32163 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.294054 32163 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:18.295912 31722 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:20:18.338055 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 226306us
I20260709 12:20:18.339855 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:18.341562 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.318774 32144 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:18.363265 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:18.351902 31934 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:18.351712 32130 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.372975 31636 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:18.384644 32130 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.396378 25710 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:18.396801 32130 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:18.399081 32096 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.001s
I20260709 12:20:18.411046 25710 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:18.394025 32100 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.434042 32130 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.434629 32130 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.437019 32130 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:18.446616 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.447655 31015 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.401487 32038 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:20:18.454627 31015 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:18.455133 31015 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.457857 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:18.460867 31015 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:18.465828 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.422853 32044 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:20:18.472944 32038 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:18.473881 32038 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:18.434392 31831 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.478744 32044 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:18.479611 32044 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.482172 32044 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.482915 31831 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.484315 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:18.489605 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2956) took 1369 ms. Trace:
I20260709 12:20:18.474255 32038 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:18.489995 25785 rpcz_store.cc:276] 0709 12:20:17.120308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.132580 (+ 12272us) service_pool.cc:224] Handling call
0709 12:20:17.132859 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:17.133057 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:20:17.138837 (+  5780us) log.cc:844] Serialized 14 byte log entry
0709 12:20:17.705308 (+566471us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:18.489088 (+783780us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b180049a920. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:20:18.489310 (+   222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:18.489356 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:18.489403 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.489462 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.489554 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127135,"prepare.queue_time_us":14825,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"spinlock_wait_cycles":513515648,"thread_start_us":69515,"threads_started":1,"wal-append.queue_time_us":195549}
I20260709 12:20:18.491927 31728 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:18.492421 31728 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.492813 25751 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:18.494647 32136 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:18.495124 32136 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.497059 32136 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:18.497545 32136 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:18.498971 32168 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.499614 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2456) took 1175 ms. Trace:
I20260709 12:20:18.494232 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:18.501107 32148 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:18.500190 25783 rpcz_store.cc:276] 0709 12:20:17.323800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.326674 (+  2874us) service_pool.cc:224] Handling call
0709 12:20:17.327091 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:17.327379 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:20:17.388391 (+ 61012us) log.cc:844] Serialized 14 byte log entry
0709 12:20:18.499154 (+1110763us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:18.499230 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:18.499296 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.499389 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.499538 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1030128,"prepare.queue_time_us":60801,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":1031985,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":80160,"threads_started":1,"wal-append.queue_time_us":283747}
I20260709 12:20:18.446225 32197 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.514335 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:18.514938 25629 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:18.518946 32197 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.491142 32038 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.520126 32038 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:18.521623 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:20:18.522981 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.523507 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:18.525472 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:18.526021 25634 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:18.528079 25637 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:18.544059 31672 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.544971 25629 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
W20260709 12:20:18.546090 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.547240 32012 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.548133 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:18.548982 25704 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:18.549508 32012 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.551251 26308 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:18.553210 32197 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:18.553756 32197 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:18.555125 32197 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.556855 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:18.546437 31672 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.566412 25635 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:18.568427 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:18.535176 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 166318us
I20260709 12:20:18.575495 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:18.576618 25707 raft_consensus.cc:2393] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:18.576787 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:18.577693 25705 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:18.581702 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:18.593182 25704 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:20:18.594287 25633 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:20:18.594942 25633 pending_rounds.cc:85] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 13
I20260709 12:20:18.595234 25633 pending_rounds.cc:107] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:20:18.595523 25633 raft_consensus.cc:2889] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:18.529788 32195 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.259s	user 0.003s	sys 0.000s
I20260709 12:20:18.599820 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:18.606279 25706 raft_consensus.cc:1275] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:20:18.610340 32100 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:18.494338 31728 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:18.595494 25603 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:18.614599 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.602314 31917 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:18.619163 26337 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:18.645460 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:18.647015 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 125487us
I20260709 12:20:18.621829 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:18.623277 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:18.646140 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:18.626570 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:18.603571 31722 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:18.601379 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:18.632715 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:18.659894 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:18.661731 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 60434.2us
I20260709 12:20:18.690865 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:18.694198 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48155.2us
I20260709 12:20:18.644517 31781 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.649410 32184 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:20:18.732471 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 71245.2us
I20260709 12:20:18.665544 32091 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.694605 32170 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:20:18.624328 32180 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.750136 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:18.631214 32181 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.753434 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:18.754331 26337 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:18.655786 32150 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:18.665071 32166 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:18.694136 32044 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.701452 32099 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:20:18.766109 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 118185us
I20260709 12:20:18.747918 32091 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:18.764896 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 102266us
I20260709 12:20:18.757056 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:18.772488 32168 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.772894 32168 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.773229 32168 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.773589 32091 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:18.773734 32055 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:18.770172 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:18.777642 25676 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.67:41061 timed out after 1.604s (SENT) [suppressed 9 similar messages]
I20260709 12:20:18.780043 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:18.791819 25751 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:18.801235 32168 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.733914 32057 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:18.768111 32148 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
W20260709 12:20:18.827418 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 56772us
I20260709 12:20:18.852892 31739 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:18.853443 31739 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.828219 25712 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 20: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:18.829170 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:18.834882 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:18.733716 32179 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:18.841199 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61226.1us
I20260709 12:20:18.694913 31722 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.698644 31831 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:18.855690 26343 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:20:18.655458 31917 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.756122 32184 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:18.766335 32099 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.861040 25785 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:18.827498 32141 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:18.854171 32012 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.903894 31015 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.751686 31781 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.912631 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:18.912875 26339 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:18.908234 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:18.768208 32150 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:18.925560 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:19.066782 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.067926 32150 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:19.068857 32214 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:20:19.074635 32173 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:19.075234 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 167088us
I20260709 12:20:18.929174 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:19.076570 25779 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:20:18.945587 32012 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:18.910235 31831 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:18.976861 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:18.859848 32057 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:18.967267 32217 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:20:19.086114 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57968 (request call id 3352) took 1001 ms. Trace:
I20260709 12:20:18.949013 26336 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:18.963992 31781 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:18.928938 32167 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.827481 32044 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:18.928180 32141 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:18.916182 32099 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:18.995612 25533 catalog_manager.cc:5697] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 10 to 14, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 14 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.124788 25533 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 10 to 13, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:19.127107 25533 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.128973 25533 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.131017 25533 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 10 to 14, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 14 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.132966 25533 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 7 to 11, term changed from 13 to 15, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68), VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 15 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:18.899560 31728 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:18.930960 31015 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:19.134987 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.136034 31015 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:18.973635 25786 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:19.078886 32224 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:19.139287 32224 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.141830 32224 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:18.986555 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 208736us
W20260709 12:20:18.996176 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 222872us
I20260709 12:20:19.143467 31917 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:19.033864 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 338009us
I20260709 12:20:19.146836 31831 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:19.146910 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:19.148545 25752 leader_election.cc:400] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:20:19.148928 25752 leader_election.cc:403] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:20:19.038854 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 406634us
I20260709 12:20:19.043668 31722 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:19.020744 32225 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:19.039539 32008 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:20:19.055914 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 135819us
I20260709 12:20:18.962160 25633 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
W20260709 12:20:19.065750 25531 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 303) took 58482 ms (client timeout 15000 ms). Trace:
I20260709 12:20:19.026423 32223 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:19.080515 31739 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:19.080217 31734 consensus_queue.cc:1048] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:19.081120 32168 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:19.074757 32180 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:19.164816 25786 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
W20260709 12:20:19.082854 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.052595 32181 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:19.086516 25712 rpcz_store.cc:276] 0709 12:20:18.084770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.127325 (+ 42555us) service_pool.cc:224] Handling call
0709 12:20:19.085919 (+958594us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54002ec2d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:19.086053 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574554496}
I20260709 12:20:19.114615 25779 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:20:19.082926 32012 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.905165 32179 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.019794 26341 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:19.151626 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:19.183320 25637 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:19.153227 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:19.123560 32141 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:19.161396 25752 leader_election.cc:400] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:19.161914 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:19.162251 25601 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:19.187352 25752 leader_election.cc:403] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:20:19.163847 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.187618 25630 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:19.162786 32223 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.013860 32184 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:19.184516 25634 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:19.158720 25636 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:20:19.160028 32225 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:19.160693 32224 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:19.183235 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:19.206451 25533 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 11, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.165225 32136 raft_consensus.cc:3060] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:19.191236 26343 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:20:19.191812 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 357596us
I20260709 12:20:19.168157 25780 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:19.193593 26339 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:20:19.194131 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 299283us
I20260709 12:20:19.196832 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:19.165395 32130 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:19.160279 32055 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:20:19.216236 25712 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:19.217365 25708 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:19.220896 25533 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 15, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 15 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:19.160051 31595 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:20:19.224977 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:19.222749 26338 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.007s	sys 0.028s
W20260709 12:20:19.225947 26338 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.007s	sys 0.028s
I20260709 12:20:19.226824 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:19.227459 25712 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:19.230609 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:19.203486 31728 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:19.233210 25533 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 11, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.193082 32175 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:19.234117 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:19.234908 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.235433 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.236138 25787 raft_consensus.cc:2393] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:19.236006 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.239567 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.240366 25781 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:19.152909 32167 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.206929 32144 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:19.269088 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:19.244956 25785 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:19.245223 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:19.271461 25708 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:19.269896 31015 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:20:19.272892 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.244227 25637 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:19.221351 32197 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:19.261296 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:19.275988 31015 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:19.284909 31015 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:19.285359 31015 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:19.277135 32214 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:20:19.221285 32152 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:19.292666 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:19.276157 25677 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:19.277837 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:19.280699 25533 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 7 to 9, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:19.221246 31996 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:20:19.243213 32181 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:19.220793 32168 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:20:19.259295 32146 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:19.308373 25533 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.258424 32224 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:19.274718 32232 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:19.228868 32130 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:19.304713 32221 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.282s	user 0.001s	sys 0.003s
I20260709 12:20:19.331882 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:19.336409 25533 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.303835 32144 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:19.301074 32197 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:19.360455 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46233.3us
I20260709 12:20:19.365147 25787 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:19.344378 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.306437 32181 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:19.277715 32175 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.330946 32037 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:19.384466 25706 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:19.390829 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.391481 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:19.394080 25712 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:19.330955 32180 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:19.390079 25603 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:19.302059 32152 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.417171 25787 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
W20260709 12:20:19.386018 32214 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:20:19.426160 25533 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 13, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:19.421205 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:19.423815 25778 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:19.384140 32181 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:19.338502 32176 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 12:20:19.330442 32233 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:20:19.449287 25533 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 11, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.331081 32224 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.460242 26344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:19.380959 32177 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:20:19.461949 25533 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 11, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 11 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:19.420210 32100 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:19.328823 32146 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.425631 32180 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:19.390599 32232 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:19.458209 32210 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:20:19.301649 32225 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.448947 31994 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:19.498045 25533 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 14. New cstate: current_term: 14 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:19.508708 32239 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:20:19.502599 31015 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.535081 25533 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 14, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 14 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:19.562987 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 269960us
I20260709 12:20:19.568635 25533 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 5 to 13, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 13 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:19.507690 32179 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.588776 26344 raft_consensus.cc:606] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:19.528961 31636 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:19.596066 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:19.606500 25630 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:19.536885 31994 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:19.538568 32100 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.501165 32236 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 12:20:19.561529 32223 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.616890 31015 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.595409 32044 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.612607 31994 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:19.519052 32144 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.634670 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:19.638154 25630 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:19.526935 32197 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.657331 25677 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:19.619484 32100 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.647986 32175 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:19.668684 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 304158us
I20260709 12:20:19.674470 32232 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.662933 32179 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:19.677379 31015 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:19.663375 32130 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:19.678637 25601 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:19.686626 31015 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:19.687182 31015 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 20 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:19.730291 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51821.2us
I20260709 12:20:19.732412 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:20:19.675112 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:19.682425 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:19.752030 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.625456 32180 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:19.688086 31015 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:19.770890 31672 raft_consensus.cc:2764] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 12:20:19.776749 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 355665us
I20260709 12:20:19.778965 25787 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:19.682853 32179 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:19.683246 32167 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.783430 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.783114 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:19.790062 25787 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:19.684826 32152 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:19.794037 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 373100us
I20260709 12:20:19.799969 25603 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:19.801084 25787 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:19.803378 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.769169 32224 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.789453 25785 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:19.788968 32179 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:20:19.809525 32229 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:20:19.781971 32168 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:19.813091 32152 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:19.829207 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.807222 32146 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:19.843588 26307 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:19.882807 31672 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.882968 32268 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:19.883746 32268 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:19.883541 31672 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:19.868438 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:19.884900 32268 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:19.885993 31672 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:19.870724 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 70789.4us
W20260709 12:20:19.887974 32244 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:20:19.894730 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:19.895535 25712 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:19.897390 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:19.898092 25708 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:19.898773 32240 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:20:19.903467 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2983) took 1070 ms. Trace:
I20260709 12:20:19.904412 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:19.905148 25707 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:19.903857 25782 rpcz_store.cc:276] 0709 12:20:18.833538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.833658 (+   120us) service_pool.cc:224] Handling call
0709 12:20:18.833943 (+   285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:19.034834 (+200891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:19.034881 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:19.047755 (+ 12874us) log.cc:844] Serialized 52 byte log entry
0709 12:20:19.509750 (+461995us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:19.794551 (+284801us) log.cc:844] Serialized 14 byte log entry
0709 12:20:19.903030 (+108479us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b5400249620. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b590a733 00007f29b598fdbf 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c
0709 12:20:19.903081 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.903155 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:19.903198 (+    43us) raft_consensus.cc:1685] finished
0709 12:20:19.903289 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:19.903410 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154087,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":4084,"spinlock_wait_cycles":73378048,"thread_start_us":893338,"threads_started":3,"wal-append.queue_time_us":568656}
W20260709 12:20:19.909958 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 66482us
I20260709 12:20:19.912566 32218 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:19.916064 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:19.918099 25634 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:19.924252 25708 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
W20260709 12:20:19.944424 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 316653us
W20260709 12:20:19.948891 32242 log.cc:927] Time spent T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:20:19.882968 32100 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:19.961620 25603 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:19.983661 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:19.984539 31015 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:19.985077 31015 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:19.985482 31015 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:19.987079 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:19.952775 32153 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:19.988502 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:19.989403 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:19.990164 25634 raft_consensus.cc:2368] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:19.880270 32168 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:19.914026 32218 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:20:19.992261 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:19.994000 25629 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:20:19.164418 25531 rpcz_store.cc:269] 0709 12:19:20.583051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.583318 (+   267us) service_pool.cc:224] Handling call
0709 12:20:19.065616 (+58482298us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:12.604498 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:12.605214 (+   716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:12.605271 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:20:12.605305 (+    34us) tablet.cc:662] Decoding operations
0709 12:20:12.605978 (+   673us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:12.606084 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:12.606114 (+    30us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:20:12.613738 (+  7624us) tablet.cc:701] Row locks acquired
0709 12:20:12.613790 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:20:12.613977 (+   187us) write_op.cc:270] Start()
0709 12:20:12.614128 (+   151us) write_op.cc:276] Timestamp: P: 1783599612613897 usec, L: 0
0709 12:20:12.614169 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:20:12.614967 (+   798us) log.cc:844] Serialized 1312 byte log entry
0709 12:20:13.387159 (+772192us) op_driver.cc:464] REPLICATION: finished
0709 12:20:13.830422 (+443263us) write_op.cc:301] APPLY: starting
0709 12:20:13.846583 (+ 16161us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:13.849906 (+  3323us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:13.850068 (+   162us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:14.040884 (+190816us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:14.041223 (+   339us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 12:20:14.041285 (+    62us) write_op.cc:312] APPLY: finished
0709 12:20:14.041608 (+   323us) log.cc:844] Serialized 41 byte log entry
0709 12:20:14.041815 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:14.041924 (+   109us) write_op.cc:454] Released schema lock
0709 12:20:14.041993 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:14.042501 (+   508us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:14.042529 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31846,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":406918,"apply.run_cpu_time_us":9710,"apply.run_wall_time_us":227587,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16685,"num_ops":3,"prepare.queue_time_us":293656,"prepare.run_cpu_time_us":27276,"prepare.run_wall_time_us":108984,"raft.queue_time_us":364457,"raft.run_cpu_time_us":27049,"raft.run_wall_time_us":206941,"replication_time_us":772817,"thread_start_us":809998,"threads_started":4,"wal-append.queue_time_us":182473,"wal-append.run_cpu_time_us":86062,"wal-append.run_wall_time_us":2265368}]]}
I20260709 12:20:20.004586 31994 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.011475 31015 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.012838 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:20.013919 32012 consensus_meta.cc:287] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.824s	user 0.013s	sys 0.000s
I20260709 12:20:20.017083 25632 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:20.027637 31595 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.019862 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.026010 25631 raft_consensus.cc:2368] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
W20260709 12:20:20.029029 31728 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.000s
I20260709 12:20:20.027878 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 1091) took 1430 ms. Trace:
I20260709 12:20:20.029446 32252 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:20.030426 32252 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.029678 26340 rpcz_store.cc:276] 0709 12:20:18.597583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.597761 (+   178us) service_pool.cc:224] Handling call
0709 12:20:18.598204 (+   443us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:18.624537 (+ 26333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.624598 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:18.654318 (+ 29720us) log.cc:844] Serialized 52 byte log entry
0709 12:20:19.226761 (+572443us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:19.261835 (+ 35074us) log.cc:844] Serialized 14 byte log entry
0709 12:20:19.281302 (+ 19467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.281399 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:20.027663 (+746264us) raft_consensus.cc:1685] finished
0709 12:20:20.027685 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.027816 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24944,"prepare.queue_time_us":30970,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":16846,"raft.queue_time_us":25277,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":6131,"spinlock_wait_cycles":12538240,"thread_start_us":467353,"threads_started":1,"wal-append.queue_time_us":891040}
W20260709 12:20:20.032409 32141 consensus_meta.cc:287] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.000s
I20260709 12:20:20.032822 32141 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.033794 32252 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.035008 32141 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:20.035645 25781 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:20.020766 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.036072 31015 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.028456 31595 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.034195 31994 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.036726 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.037907 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:20.038089 31595 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.038815 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:20.030438 32012 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.039337 25709 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:20.039633 25635 raft_consensus.cc:1240] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:20.041246 32012 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.029465 31728 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18.
I20260709 12:20:20.038475 25708 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:20.050716 31595 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.051265 31595 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.053382 31595 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.054064 25708 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:20.054510 32175 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.055162 32057 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.055097 32175 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.055765 32057 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:20.055955 25709 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:20.056278 32057 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.057255 25603 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:20.057611 32175 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.058140 32238 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:20.058811 32238 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:20.059844 32238 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.064621 25603 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:19.984869 32136 consensus_meta.cc:287] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.763s	user 0.013s	sys 0.000s
I20260709 12:20:20.077247 25781 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:19.923914 32152 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.017792 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:20.085626 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 67928.3us
W20260709 12:20:20.088011 32265 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:20:20.089715 31636 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:20.090806 32152 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.058434 32057 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.091549 32267 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:20.092231 32267 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:20.093204 32267 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.018303 32074 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:20:20.107718 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 147457us
W20260709 12:20:20.111284 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:20.114203 25750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.023s	sys 0.024s
W20260709 12:20:20.114748 25750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.024s	sys 0.024s
W20260709 12:20:20.115566 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 550622us
I20260709 12:20:20.116653 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.117419 25782 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:20.084667 32095 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:20.086256 32136 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 12. Current term is 13.
I20260709 12:20:19.988554 32153 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.142683 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.143697 25782 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:20.109479 32074 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:20.110724 31636 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:20.161486 31636 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.163946 31636 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:20.152036 32091 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:20.164673 32091 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.159577 32259 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:20.159830 32168 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.170112 31917 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:20.171054 31917 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.176741 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:20.169363 32099 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 1.008s	user 0.009s	sys 0.004s
I20260709 12:20:20.152562 32074 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:20.178690 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:20.179704 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 229857us
W20260709 12:20:20.189087 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.186452 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.174242 31917 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.189858 25782 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:20.119616 31734 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:20:20.147571 32141 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:20.195461 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.208410 25778 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:20:20.151157 32095 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:20.211512 32153 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.220634 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.234119 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.243798 25782 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:20.190812 32099 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.248221 32099 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:20.248632 32237 log.cc:927] Time spent T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.746s	user 0.002s	sys 0.001s
I20260709 12:20:20.249390 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:20.250020 25783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 2985) took 1406 ms. Trace:
I20260709 12:20:20.250121 26340 raft_consensus.cc:606] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:20.250694 25783 rpcz_store.cc:276] 0709 12:20:18.843598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.843708 (+   110us) service_pool.cc:224] Handling call
0709 12:20:18.844037 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:18.849122 (+  5085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.849150 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:18.859328 (+ 10178us) log.cc:844] Serialized 52 byte log entry
0709 12:20:19.154012 (+294684us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:20:19.154055 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.154123 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:20.247961 (+1093838us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:20.249720 (+  1759us) raft_consensus.cc:1685] finished
0709 12:20:20.249741 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.249957 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6510,"raft.queue_time_us":153947,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":5921,"spinlock_wait_cycles":25692032,"thread_start_us":208664,"threads_started":1,"wal-append.queue_time_us":477871}
I20260709 12:20:20.258786 32091 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.266307 25784 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:20.270210 25778 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:20.278872 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:20.281177 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:20.282226 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:20.284731 32272 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 12:20:20.286077 32012 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:20:20.287190 32275 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:20.289613 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.289865 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.170213 32259 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:20.273754 32181 consensus_meta.cc:287] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.004s
I20260709 12:20:20.258328 32225 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:20.265731 32096 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:20.307945 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.319166 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.321223 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.322077 25709 raft_consensus.cc:2368] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
I20260709 12:20:20.226157 32141 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.324190 26341 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
W20260709 12:20:20.324968 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.327463 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.328377 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.332208 25637 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:20.333815 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.334425 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:20.336030 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.334473 26340 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:20.338909 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.339478 26340 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:20.342653 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.342857 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:20.345095 26340 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:20:20.345495 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.346269 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.352988 26336 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:20.360610 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.384945 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.391415 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:20.407339 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:20.331995 32230 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.410799 25709 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:20.413352 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:20.305743 32181 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.349486 32212 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.419963 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.353515 31734 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.437799 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.434445 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:20.323632 32096 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.420882 25710 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:20:20.441025 25633 raft_consensus.cc:1240] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:20.444415 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.445417 25709 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:20.421775 32146 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:20.451538 25709 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:20.433596 32008 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:20.439419 32278 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:20.421034 32271 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:20.332298 32095 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.426641 32230 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.486183 31994 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:20:20.460040 32146 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:20.467504 31917 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.484067 32268 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:20.561805 32268 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:20.468616 32008 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.562391 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.562696 32008 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:20.542903 32274 log.cc:927] Time spent T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:20:20.543434 32141 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:20.543915 32280 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:20:20.568048 31917 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.544925 32096 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:20.441037 31734 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.569999 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.570885 25709 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:20.572011 31734 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:20.548339 32181 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.538956 32146 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:20.575419 31917 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:20.542388 32273 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:20:20.489351 32278 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.581843 32278 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:20.484539 32271 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:20.582127 25750 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:20.582453 32271 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:20.583249 25635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60736 (request call id 1480) took 1464 ms. Trace:
I20260709 12:20:20.583629 25635 rpcz_store.cc:276] 0709 12:20:19.119115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.120204 (+  1089us) service_pool.cc:224] Handling call
0709 12:20:19.120577 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:20.039972 (+919395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:20.583011 (+543039us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400363654. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:20.583187 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113287424}
I20260709 12:20:20.585145 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:20.586686 32098 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:20.587132 32098 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:20.587571 32098 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:20.587826 25713 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:20.590690 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.592260 25783 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:20.593052 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.551532 32259 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.597249 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:20.436368 32212 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.556047 25782 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:20:20.604979 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.605786 25782 raft_consensus.cc:2368] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:20:20.556172 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:20.558674 32179 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.678s	user 0.006s	sys 0.006s
I20260709 12:20:20.560520 32230 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.578106 25709 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:20.594954 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.562316 32268 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:20.610349 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.627858 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:20.628468 25676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.013s	sys 0.024s
I20260709 12:20:20.620136 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.625769 25778 raft_consensus.cc:2468] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:20.636328 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.636551 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.637640 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.637979 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.638259 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.639003 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.639222 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.639983 25778 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:20.640934 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.615808 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:20.642511 25787 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:20:20.642714 25676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.014s	sys 0.024s
W20260709 12:20:20.643536 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 539182us
I20260709 12:20:20.645723 25787 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:20.647063 25752 leader_election.cc:400] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
I20260709 12:20:20.647470 25752 leader_election.cc:403] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:20.628592 25782 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:20.650012 25787 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:20.651597 25787 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:20:20.620585 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.653849 25782 raft_consensus.cc:1275] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:20:20.655439 25781 raft_consensus.cc:1240] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 16. Ops: []
I20260709 12:20:20.631232 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.632072 25751 leader_election.cc:400] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
W20260709 12:20:20.630416 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.659323 25751 leader_election.cc:403] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:20:20.634784 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 200911us
I20260709 12:20:20.659425 25710 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:20.666172 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.598206 31994 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:20.668650 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:20.669930 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:20.671744 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:20.675872 25707 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 19: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:20.677407 25752 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:20.680264 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.680931 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:20.679448 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.684809 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:20.682997 26336 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:20.692127 31739 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:20.697568 25712 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:20.702221 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:20.657259 32141 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:20.721004 32285 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.178s	user 0.002s	sys 0.000s
I20260709 12:20:20.679404 31996 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
I20260709 12:20:20.729480 32167 consensus_queue.cc:1059] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:20.729825 32284 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:20.715117 32091 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:20.731686 32212 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.715849 32291 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.625471 32251 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:20:20.677614 32230 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 12:20:20.779156 25784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50982 (request call id 2980) took 1946 ms (client timeout 2133 ms). Trace:
W20260709 12:20:20.779680 25784 rpcz_store.cc:269] 0709 12:20:18.832619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.834554 (+  1935us) service_pool.cc:224] Handling call
0709 12:20:20.778883 (+1944329us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001dd3d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:20.779077 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3182421760}
I20260709 12:20:20.686885 32148 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.781193 25780 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:20:20.783080 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1247) took 1547 ms. Trace:
I20260709 12:20:20.783540 25780 rpcz_store.cc:276] 0709 12:20:19.235970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.237138 (+  1168us) service_pool.cc:224] Handling call
0709 12:20:20.780925 (+1543787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:20.781755 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:20.782818 (+  1063us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001dd3d0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:20.782990 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3549567744}
I20260709 12:20:20.630059 32179 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.672657 32168 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:20:20.787185 32179 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.779217 32291 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:20.741151 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 54709.7us
I20260709 12:20:20.788154 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:20.788853 25708 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
W20260709 12:20:20.744334 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 48887.9us
I20260709 12:20:20.692961 32150 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:20.745395 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 258517us
I20260709 12:20:20.799779 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:20.804219 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:20.765805 32268 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.768256 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:20.731742 32008 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.685743 31994 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.767297 31981 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:20.771634 32018 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:20:20.775313 32144 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:20.737577 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:20.756938 32250 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:20:20.840638 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.840962 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.842150 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.815665 26338 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:20.827374 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.843205 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.822429 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.828733 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.830376 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.845451 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.845651 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:20.846602 26335 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:20.834438 25787 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:20:20.828819 32074 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.001s
I20260709 12:20:20.848883 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.862234 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:20.866324 26337 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:20.869129 25782 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:20.828576 32218 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:20.867673 26339 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:20.873868 25710 raft_consensus.cc:2368] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19.
W20260709 12:20:20.876749 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 188451us
I20260709 12:20:20.881810 26342 raft_consensus.cc:2393] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:20.768842 32141 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.812987 32091 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:20.831158 31739 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:20.893287 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.894899 26336 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 11.
I20260709 12:20:20.808518 32012 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:20:20.802253 32284 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:20.948877 32284 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:20.889226 32226 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:20.893286 31739 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:20.950150 31739 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:20.873629 32233 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:20:20.954063 32279 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:20:20.922056 32100 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.849612 32144 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:20.958703 32298 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:20:20.961006 25781 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:20:20.945492 32008 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.882412 32150 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:20:20.955679 32100 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:20.939419 32291 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:20.941039 31994 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:20.942966 32141 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:20.962625 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.962759 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.964213 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.966836 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:20.958269 32144 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:20.983408 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.987399 25633 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:20.878975 32074 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:20.970736 32225 consensus_meta.cc:287] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.003s
I20260709 12:20:21.017922 25779 raft_consensus.cc:2410] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:20:20.968472 32297 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:21.022454 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.976994 32148 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
W20260709 12:20:21.023437 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 285931us
I20260709 12:20:21.036556 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:21.034320 26308 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:21.036968 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:21.038069 25633 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:21.040426 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.023420 32223 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.993631 32146 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.056332 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:21.056800 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:21.056998 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:20.982398 32271 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:21.049959 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 186740us
I20260709 12:20:21.057030 25629 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.060025 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:21.063215 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:21.042907 32259 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:21.136582 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:21.137451 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 77519.4us
I20260709 12:20:21.029038 32098 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:21.074057 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.142068 25708 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:20:21.146212 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 87849.4us
I20260709 12:20:21.033891 32291 consensus_queue.cc:1059] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:20:21.150944 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:21.060026 32251 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:21.156899 25629 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:21.061987 32296 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:21.172001 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:21.160316 25786 raft_consensus.cc:2393] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:21.158972 25708 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:21.033727 32297 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:21.159413 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 102693us
W20260709 12:20:21.145473 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 95667.5us
I20260709 12:20:21.136101 32176 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:21.178853 25782 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 16. Current term is 17. Ops: []
W20260709 12:20:21.160159 32288 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:20:21.194747 25636 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:20:21.177058 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:21.196348 25751 leader_election.cc:400] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
I20260709 12:20:21.161549 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:21.185388 32305 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:20:21.187497 25705 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:20:21.216650 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 68074.5us
I20260709 12:20:21.190793 32297 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:21.191156 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:21.191882 32098 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:21.193390 32271 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:21.162773 32226 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:21.160437 32148 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:21.159891 32012 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:20:21.217782 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 56349us
I20260709 12:20:21.178923 32176 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:21.220952 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.256258 25787 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
W20260709 12:20:21.222172 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45199.8us
I20260709 12:20:21.258428 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:21.197232 32101 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:21.261806 25630 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:21.226070 32278 raft_consensus.cc:3055] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:20:21.231364 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53952.3us
I20260709 12:20:21.237238 32306 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:21.239651 32300 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.300s	user 0.002s	sys 0.000s
I20260709 12:20:21.151899 32230 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:20:21.280237 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:21.222018 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:21.058691 32223 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.249657 32148 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:21.293934 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.295351 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.295459 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.223554 32018 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:21.264341 32176 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.289795 25711 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 14: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:21.292878 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.264425 32008 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:21.269773 32278 raft_consensus.cc:740] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:21.315081 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:21.300007 32148 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:21.320379 32278 consensus_queue.cc:260] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.279805 32100 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.268172 32225 consensus_queue.cc:1059] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:20:21.293767 32259 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:21.335537 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:21.339741 25713 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:21.277702 32057 consensus_queue.cc:1059] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:20:21.317790 32074 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.279670 32037 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:20:21.336412 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.277277 32233 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:20:21.348420 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.364681 25778 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:21.313037 32271 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:21.366869 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:21.327139 32008 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:21.301306 32306 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.372956 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.373430 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:21.375162 32318 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:20:21.376876 32317 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:20:21.378310 32310 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:20:21.384104 32233 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.384654 25784 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:21.384922 32101 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:21.385430 32101 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:21.386413 32250 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:21.386996 32250 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:21.391798 31739 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.394177 25783 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:21.399883 31595 raft_consensus.cc:3055] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:21.400334 31595 raft_consensus.cc:740] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:21.401113 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:21.402165 25780 raft_consensus.cc:2393] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:21.403131 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.404008 25780 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:21.404912 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.408936 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.410363 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:21.411108 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 525963us
W20260709 12:20:21.412146 25779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50194 (request call id 1241) took 2178 ms (client timeout 1550 ms). Trace:
I20260709 12:20:21.384742 32233 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:21.387261 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.414901 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:21.415867 25633 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:21.415808 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:21.412485 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:21.416368 25778 raft_consensus.cc:2376] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18. Re-sending same reply.
I20260709 12:20:21.416810 25784 raft_consensus.cc:2368] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18.
W20260709 12:20:21.412695 25779 rpcz_store.cc:269] 0709 12:20:19.234052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.234674 (+   622us) service_pool.cc:224] Handling call
0709 12:20:21.411934 (+2177260us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b540030af54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:21.412068 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4097984384}
I20260709 12:20:21.417338 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.417651 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.417816 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.418588 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.419130 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.418064 25778 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:21.420322 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:21.421257 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:21.422864 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.405589 25780 raft_consensus.cc:2376] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18. Re-sending same reply.
I20260709 12:20:21.419595 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:21.421963 26336 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:21.429996 25676 leader_election.cc:400] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:20:21.430361 25676 leader_election.cc:403] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:20:21.431448 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:21.431957 25778 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:21.434314 25751 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:21.435469 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.435900 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:21.436163 26342 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:21.438221 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:21.438704 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.451961 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:21.459326 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:21.462126 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:21.434100 32184 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:21.346560 32259 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.436524 31996 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:21.437126 32098 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:21.463994 32299 log.cc:927] Time spent T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.526s	user 0.000s	sys 0.006s
I20260709 12:20:21.439550 32168 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 10. Current term is 11.
I20260709 12:20:21.479535 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:21.372522 32271 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:21.475855 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 96487.3us
I20260709 12:20:21.470803 32012 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.466938 32326 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:21.454944 32284 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:21.437726 32232 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:21.470268 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:21.513136 25780 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:21.466769 32283 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:21.466509 32130 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:21.485067 31015 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:21.401171 31595 consensus_queue.cc:260] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.438630 32180 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:20:21.503629 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.476626 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:21.468627 32309 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:20:21.526934 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50391.1us
I20260709 12:20:21.527891 32141 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:21.527987 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:21.529631 32326 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:21.530412 32326 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:21.533341 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.539762 25603 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:21.504086 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:21.453934 32074 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:20:21.463339 32238 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:20:21.511579 31734 raft_consensus.cc:993] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:20:21.478461 31994 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:20:21.475823 32095 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:20:21.549082 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:21.554003 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:21.555536 31015 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:21.560022 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 94704us
I20260709 12:20:21.511909 32012 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:21.571584 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:21.572535 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:21.500902 32008 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.576593 31015 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:21.577039 31015 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:21.577410 31015 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.563005 31722 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:21.553836 32233 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:21.576171 32291 raft_consensus.cc:1081] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
W20260709 12:20:21.581636 32218 consensus_meta.cc:287] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.685s	user 0.013s	sys 0.000s
I20260709 12:20:21.596052 32179 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:21.571450 32314 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:20:21.605683 31739 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:21.606891 32306 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:21.608594 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.595007 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:21.552210 32148 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.618660 31015 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.627038 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.633598 25786 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:21.574029 32037 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:20:21.588471 31722 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.654853 31722 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:21.655704 32332 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:20:21.589144 32168 consensus_queue.cc:1048] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:20:21.648783 25629 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:21.649751 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.652499 32311 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:21.652706 32180 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:21.653625 31831 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:20:21.654047 32037 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:21.635466 32218 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:21.588018 32284 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:20:21.667450 31015 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:21.672046 25787 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:21.587772 32333 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:20:21.671394 32259 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.677220 32184 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:21.698539 31722 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.703099 31722 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.704191 32148 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:21.706027 32271 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:21.707417 32176 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.709821 32176 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:21.686875 32268 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.714709 32012 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.714938 31898 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:21.715644 31898 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:21.716987 31898 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.720305 31898 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:21.772333 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 177136us
I20260709 12:20:21.774829 25676 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:21.779983 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.785759 25676 leader_election.cc:400] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18.
I20260709 12:20:21.786384 25676 leader_election.cc:403] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:20:21.789238 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:21.807116 31722 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.809633 31722 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.828965 32271 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:21.829998 32271 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:21.833529 32271 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:21.835318 32057 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18.
I20260709 12:20:21.839032 32008 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:21.839334 32283 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:21.844532 31595 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:21.853039 32223 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:21.854321 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 207635us
I20260709 12:20:21.856990 31722 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.857388 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:21.860051 32100 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:21.860622 32100 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:21.861116 32100 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:21.863477 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:21.864542 26339 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.867471 32173 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.872666 32173 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:21.868990 25753 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.883147 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:21.888401 25781 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.890241 32173 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:21.871739 32130 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:21.890930 32130 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:21.892997 32130 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.896195 32130 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:21.896965 32100 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.874150 32146 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:21.898769 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 109844us
I20260709 12:20:21.899920 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:21.864661 31722 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:20:21.906870 32167 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:21.909351 31722 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:21.909997 31722 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:21.844551 32259 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:21.926689 26339 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:21.939796 32294 raft_consensus.cc:3055] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:20:21.940196 32316 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.706s	user 0.001s	sys 0.001s
I20260709 12:20:21.940812 32294 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:21.943136 25709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3404) took 1151 ms. Trace:
I20260709 12:20:21.942770 32294 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:21.919008 32271 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.950363 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:21.951330 26342 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:21.959623 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61358.7us
I20260709 12:20:21.964113 32259 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:21.943614 25709 rpcz_store.cc:276] 0709 12:20:20.791486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.791680 (+   194us) service_pool.cc:224] Handling call
0709 12:20:20.792192 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:20.793441 (+  1249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:20.793489 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:20.802157 (+  8668us) log.cc:844] Serialized 26 byte log entry
0709 12:20:21.147008 (+344851us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:21.147044 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.147108 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:21.932215 (+785107us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:21.942311 (+ 10096us) raft_consensus.cc:1685] finished
0709 12:20:21.942345 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.942853 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2896,"raft.queue_time_us":773,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10324,"spinlock_wait_cycles":32625920,"thread_start_us":344492,"threads_started":1,"wal-append.queue_time_us":425193}
I20260709 12:20:21.966785 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:21.967290 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.968379 25629 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:21.971395 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.971838 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:21.975063 32197 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:21.978859 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.979707 25635 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
W20260709 12:20:21.982932 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.984651 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:21.987028 32101 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:21.992079 32278 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:21.992967 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:22.005698 32259 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:22.007717 32259 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:20:22.008023 32301 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 1.076s	user 0.000s	sys 0.004s
I20260709 12:20:22.009909 32294 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:22.011005 32101 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:22.012058 25785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1264) took 1423 ms. Trace:
I20260709 12:20:22.012414 25785 rpcz_store.cc:276] 0709 12:20:20.588349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.591339 (+  2990us) service_pool.cc:224] Handling call
0709 12:20:20.591602 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:20.597310 (+  5708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:20.597339 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:20.618999 (+ 21660us) log.cc:844] Serialized 26 byte log entry
0709 12:20:20.793841 (+174842us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:20.793879 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:20.793941 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:22.007068 (+1213127us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:22.011494 (+  4426us) raft_consensus.cc:1685] finished
0709 12:20:22.011515 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:22.011834 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5362,"raft.queue_time_us":6224,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":21952,"spinlock_wait_cycles":74145152,"thread_start_us":116110,"threads_started":1,"wal-append.queue_time_us":277553}
I20260709 12:20:22.019858 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:22.021363 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 74066.8us
I20260709 12:20:22.022516 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:22.024096 32144 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:22.024757 32144 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:22.020313 32271 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:22.021087 32304 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:22.022230 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 458171us
I20260709 12:20:22.033859 32167 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:22.026298 32144 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.009023 32259 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.047031 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:22.055550 25782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 3016) took 1176 ms. Trace:
I20260709 12:20:22.056035 25782 rpcz_store.cc:276] 0709 12:20:20.879530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.886678 (+  7148us) service_pool.cc:224] Handling call
0709 12:20:20.887135 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:21.179358 (+292223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:22.054945 (+875587us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54001a86d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:22.055335 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670382848}
I20260709 12:20:22.042856 32259 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:22.066593 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.070680 32100 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:22.070796 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:22.073381 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:22.078107 25781 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:22.082968 26336 raft_consensus.cc:1240] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:22.083678 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:22.083937 25778 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:22.086031 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:22.087160 25706 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:22.087225 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:22.087867 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:22.082782 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:22.044600 31722 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.094177 32304 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:22.097095 32304 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:22.098410 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:22.100818 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:22.101102 26342 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:22.101821 26335 raft_consensus.cc:2368] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
W20260709 12:20:22.111135 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.118767 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:22.119339 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56416
I20260709 12:20:22.120117 25636 raft_consensus.cc:606] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Received request to transfer leadership
I20260709 12:20:22.126652 32271 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:22.128724 25603 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:22.130720 32311 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:20:22.130770 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 83043.6us
I20260709 12:20:22.131443 32144 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:22.132499 25601 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:22.136328 32296 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:22.120543 25636 raft_consensus.cc:614] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:22.146008 32250 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:22.137094 32296 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:22.152942 32296 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.138509 32197 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:22.156201 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:22.159860 31722 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:22.160893 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.187180 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:22.189141 25710 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:22.203284 25781 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:22.204366 32304 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:22.215126 32304 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.217502 31898 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:22.218032 31898 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.218387 31898 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:22.218703 32304 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:22.229643 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:22.230075 25635 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:22.231392 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 46943.2us
I20260709 12:20:22.232769 25750 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:22.233891 32226 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:20:22.243559 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.234457 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:22.270696 32271 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:22.243880 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:22.246845 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:22.281411 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:22.281787 32270 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:22.282555 32177 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:22.283143 32177 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:22.283475 32297 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:22.283942 32297 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:22.285079 32297 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.285400 32177 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.251299 32095 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:20:22.288372 32252 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:20:22.258550 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 110653us
I20260709 12:20:22.290762 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:22.291858 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 252265us
I20260709 12:20:22.293707 32101 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:20:22.263339 32348 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:20:22.298265 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:22.299738 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:22.300348 26342 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:20:22.311952 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.312628 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:22.313668 25782 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:22.316896 25782 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:22.318070 32197 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:22.319005 32197 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.322338 32197 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:22.326591 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:22.327666 25630 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:22.344533 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:22.345484 25782 raft_consensus.cc:2376] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18. Re-sending same reply.
I20260709 12:20:22.364179 25750 leader_election.cc:400] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
I20260709 12:20:22.364746 25750 leader_election.cc:403] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:20:22.365929 32271 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:22.366765 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 85435.3us
I20260709 12:20:22.368101 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:22.370827 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 120688us
I20260709 12:20:22.370818 32270 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:22.373536 32270 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:22.374256 32270 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.375067 32259 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.376600 26343 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:22.380481 26339 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:20:22.388535 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:22.389303 25781 raft_consensus.cc:2376] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13. Re-sending same reply.
I20260709 12:20:22.372797 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:22.373575 25636 raft_consensus.cc:1240] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: []
W20260709 12:20:22.390640 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 113920us
W20260709 12:20:22.416725 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 126029us
W20260709 12:20:22.419137 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.424608 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:22.425446 25638 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:22.397894 32297 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:22.428866 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:22.435252 26341 raft_consensus.cc:2410] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:20:22.448916 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:22.458210 25784 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:22.478524 32343 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 12:20:22.484323 25533 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 197) took 15788 ms (client timeout 15000 ms). Trace:
W20260709 12:20:22.484946 25533 rpcz_store.cc:269] 0709 12:20:06.695578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.695828 (+   250us) service_pool.cc:224] Handling call
0709 12:20:22.483995 (+15788167us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:20.153494 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:20.261569 (+108075us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:20.261652 (+    83us) write_op.cc:435] Acquired schema lock
0709 12:20:20.261683 (+    31us) tablet.cc:662] Decoding operations
0709 12:20:20.416322 (+154639us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:20.429087 (+ 12765us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:20.433975 (+  4888us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:20:20.545160 (+111185us) tablet.cc:701] Row locks acquired
0709 12:20:20.545182 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:20:20.545454 (+   272us) write_op.cc:270] Start()
0709 12:20:20.545572 (+   118us) write_op.cc:276] Timestamp: P: 1783599620545398 usec, L: 0
0709 12:20:20.545597 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:20:20.546329 (+   732us) log.cc:844] Serialized 7107 byte log entry
0709 12:20:20.777378 (+231049us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b28850ac 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b
0709 12:20:21.362098 (+584720us) op_driver.cc:464] REPLICATION: finished
0709 12:20:21.525259 (+163161us) write_op.cc:301] APPLY: starting
0709 12:20:21.525362 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:21.664995 (+139633us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:21.670434 (+  5439us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:21.727188 (+ 56754us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:21.727504 (+   316us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:20:21.727563 (+    59us) write_op.cc:312] APPLY: finished
0709 12:20:21.728312 (+   749us) log.cc:844] Serialized 164 byte log entry
0709 12:20:21.728539 (+   227us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:21.728691 (+   152us) write_op.cc:454] Released schema lock
0709 12:20:21.728744 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:21.729118 (+   374us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:21.729145 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2663687,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":151572,"apply.run_cpu_time_us":24766,"apply.run_wall_time_us":203865,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79475,"num_ops":18,"prepare.queue_time_us":293532,"prepare.run_cpu_time_us":30891,"prepare.run_wall_time_us":889334,"raft.queue_time_us":144037,"raft.run_cpu_time_us":16333,"raft.run_wall_time_us":175554,"replication_time_us":794092,"spinlock_wait_cycles":7991552,"thread_start_us":771246,"threads_started":4,"wal-append.queue_time_us":505447}]]}
W20260709 12:20:22.502841 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 84678.5us
I20260709 12:20:22.526463 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:22.494071 32271 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16.
W20260709 12:20:22.543737 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.471288 32259 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:22.510528 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79255us
I20260709 12:20:22.560683 25778 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: [14.9-18.11]
I20260709 12:20:22.514711 32312 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:22.516244 32270 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:22.558878 32226 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.502065 31898 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.566078 31595 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.567180 32312 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:22.593299 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 66965.1us
I20260709 12:20:22.570778 32146 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:22.607461 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:22.608892 31595 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.610471 32270 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:22.613359 32130 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:22.592305 32294 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.615183 32294 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.593711 32312 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:22.607596 25638 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:22.597756 32259 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.633051 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:22.632762 32259 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.633996 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:22.635010 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.637639 32259 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:22.628229 32146 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:22.640391 32294 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:22.643589 31898 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:22.655156 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:22.657012 26341 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:22.624611 32226 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.669180 32226 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:22.674793 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:22.673425 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79394.5us
I20260709 12:20:22.677384 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:22.679708 25784 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:20:22.671814 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:22.682844 25752 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:22.672113 32356 log.cc:927] Time spent T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:20:22.685913 26343 raft_consensus.cc:2410] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 12:20:22.670408 32359 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
I20260709 12:20:22.699903 32312 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:22.701296 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:22.701887 32146 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.702008 31595 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:22.694806 31898 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:22.712984 31722 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:22.755539 25630 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
W20260709 12:20:22.776438 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 93673.8us
I20260709 12:20:22.784217 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:22.786661 25535 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.773947 32250 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:22.796736 25535 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 11 to 16, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:22.802541 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.802548 31722 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:22.815495 32259 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:22.839639 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 424429us
W20260709 12:20:22.838747 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 407642us
I20260709 12:20:22.838869 32259 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:22.856701 25780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.010s	sys 0.023s
W20260709 12:20:22.861270 25780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.011s	sys 0.024s
I20260709 12:20:22.865869 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:22.866986 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:22.845381 25535 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 10 to 12, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 12 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.874882 25704 raft_consensus.cc:2410] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:20:22.879563 32197 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:22.888355 31595 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:22.888981 31595 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:22.863338 32259 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.904695 32150 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:22.905515 32150 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.910429 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.913507 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:22.921800 32150 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:22.922991 31722 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.931223 32257 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:22.931795 32257 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:22.932986 32257 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.933607 31898 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:22.934744 32312 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:22.935716 32312 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:22.936410 32312 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.945847 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.946688 25778 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:22.835528 32250 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.951195 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.952354 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:22.953258 32250 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:22.874415 32321 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:22.955114 32321 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:22.953276 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:22.934476 31898 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:22.975881 25783 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:22.985222 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.986466 25535 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:22.994028 25535 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 8 to 13, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:22.994758 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:22.997133 25778 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:22.958385 32321 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:23.001623 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.013895 25787 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
W20260709 12:20:23.019703 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:23.036008 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 423155us
I20260709 12:20:23.029824 32312 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.069103 25752 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:23.075194 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 441290us
I20260709 12:20:23.076673 25535 catalog_manager.cc:5697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 8 to 11, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 11 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:23.048563 32250 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:23.098399 32364 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
I20260709 12:20:23.095849 32312 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:23.109912 25535 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 9 to 11, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 11 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:23.147764 26337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.021s	sys 0.024s
I20260709 12:20:23.107769 32250 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.166342 25535 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:23.097206 31595 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.184134 25676 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:23.166164 32304 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:23.189635 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:23.192175 32362 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.291s	user 0.005s	sys 0.009s
W20260709 12:20:23.202418 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 359589us
I20260709 12:20:23.185360 32296 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:23.227345 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 178416us
W20260709 12:20:23.236258 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 382625us
I20260709 12:20:23.204077 31898 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:23.242578 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 360448us
I20260709 12:20:23.210440 32230 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:23.264173 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.199095 32304 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.299084 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.302065 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:23.302666 25780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50982 (request call id 3039) took 1159 ms. Trace:
W20260709 12:20:23.303578 32382 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:20:23.303853 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.242460 32375 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:23.304636 25707 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
W20260709 12:20:23.305372 32370 log.cc:927] Time spent T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:20:23.228945 32296 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.305889 32377 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:23.306591 32377 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:23.307096 32372 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:20:23.307471 32374 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:20:23.307646 32296 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:23.308629 32378 log.cc:927] Time spent T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 12:20:23.309520 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 69433.4us
I20260709 12:20:23.311296 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.312079 25706 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:23.313208 32377 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:23.315040 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.279604 32230 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.318706 32381 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.319170 32230 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:23.319301 32381 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.297322 25535 catalog_manager.cc:5697] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 10 to 11, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 11 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:23.321507 32381 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.322647 32383 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:23.303109 25780 rpcz_store.cc:276] 0709 12:20:22.143751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.143936 (+   185us) service_pool.cc:224] Handling call
0709 12:20:22.147145 (+  3209us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:22.177948 (+ 30803us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:22.177979 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:22.178767 (+   788us) log.cc:844] Serialized 52 byte log entry
0709 12:20:22.861723 (+682956us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:23.191217 (+329494us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:23.224485 (+ 33268us) log.cc:844] Serialized 14 byte log entry
0709 12:20:23.226215 (+  1730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:23.226318 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:23.302373 (+ 76055us) raft_consensus.cc:1685] finished
0709 12:20:23.302474 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:23.302613 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43282,"raft.queue_time_us":63623,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":26542,"spinlock_wait_cycles":136245504,"thread_start_us":961543,"threads_started":2,"wal-append.queue_time_us":721461}
I20260709 12:20:23.323324 32383 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.323802 32250 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:23.325268 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:23.326306 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.327236 25633 raft_consensus.cc:2393] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 20: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:23.328361 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.329728 32259 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.326447 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:23.321635 25535 catalog_manager.cc:5697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 13 to 14, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 14 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:23.319024 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 153522us
I20260709 12:20:23.335813 25778 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:23.338574 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:23.342392 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:23.359441 25781 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.330927 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.339715 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:23.361413 25635 raft_consensus.cc:2393] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:23.242749 32270 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:23.304581 32379 log.cc:927] Time spent T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:20:23.376134 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.363173 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:23.377094 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.382656 25636 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:23.330636 31595 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:23.293198 32130 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:23.384227 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1484) took 1226 ms. Trace:
I20260709 12:20:23.386253 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:23.335899 31898 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.380373 32312 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:23.401786 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:23.377648 32250 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:20:23.405535 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.404874 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:23.409562 25778 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:23.385211 32146 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:23.413545 26341 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:20:23.425056 26341 pending_rounds.cc:85] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Aborting all ops after (but not including) 14
I20260709 12:20:23.396399 31722 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.395341 32321 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:23.401201 26337 rpcz_store.cc:276] 0709 12:20:22.072712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.086229 (+ 13517us) service_pool.cc:224] Handling call
0709 12:20:22.086712 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:22.087181 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:20:22.441091 (+353910us) log.cc:844] Serialized 14 byte log entry
0709 12:20:23.291484 (+850393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:23.291579 (+    95us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:23.291651 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:23.291881 (+   230us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:23.299186 (+  7305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12382,"prepare.queue_time_us":12700,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":174064,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":78,"thread_start_us":1076343,"threads_started":3,"wal-append.queue_time_us":720943}
I20260709 12:20:23.415220 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:23.435395 32146 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.431257 32297 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.411724 32294 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.435755 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.410377 32226 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:23.410197 32395 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:20:23.425369 26341 pending_rounds.cc:107] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:20:23.443740 26341 raft_consensus.cc:2889] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:23.417416 31898 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.414829 32312 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:23.449806 32259 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:23.384835 32270 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.435561 32376 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.433697 32321 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:23.384261 25784 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:20:23.478933 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 101037us
I20260709 12:20:23.450452 32391 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:20:23.492826 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 278650us
I20260709 12:20:23.461845 32312 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.504067 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:23.472295 32321 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.511832 26335 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:23.443483 31722 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.449018 32294 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.455700 32297 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.520080 26343 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:23.485615 32391 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:23.493850 32259 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:23.501163 32074 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.013s	sys 0.011s
W20260709 12:20:23.535665 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:23.551124 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 153771us
W20260709 12:20:23.562640 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 148740us
I20260709 12:20:23.587335 26341 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:23.593508 25636 raft_consensus.cc:1240] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:23.591972 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.494287 32226 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.602794 25638 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:23.596987 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:23.597592 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.604357 26335 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:23.598197 25535 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 6 to 9, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:20:23.536981 32074 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.013s	sys 0.012s
I20260709 12:20:23.604521 25630 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:23.596043 25711 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:23.527172 32296 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.615294 25630 raft_consensus.cc:2410] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:20:23.625800 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 127582us
I20260709 12:20:23.630151 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:23.632961 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:23.647034 25781 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
I20260709 12:20:23.607622 32399 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:23.588984 32259 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:23.651582 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53529.4us
W20260709 12:20:23.649412 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 94306.8us
I20260709 12:20:23.644045 32268 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.645283 32376 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:23.636358 32304 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.659067 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:23.664027 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50741.5us
I20260709 12:20:23.681813 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.682015 32268 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.682715 25631 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:23.652704 32146 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:23.652545 32387 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.695048 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.696254 32394 consensus_queue.cc:1059] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:20:23.697110 32394 consensus_queue.cc:1243] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:20:23.697621 32394 raft_consensus.cc:3055] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:23.697955 32394 raft_consensus.cc:740] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:23.701279 32394 consensus_queue.cc:260] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.702736 32394 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:23.668946 32383 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:23.707597 25635 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:23.652261 32296 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.711675 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:23.714511 32197 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:23.742899 32197 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.714852 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:23.719249 32400 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:20:23.720072 32412 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:20:23.721146 31898 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.722386 32417 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:23.723784 32294 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:23.724713 32402 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:20:23.724829 32268 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.727960 32270 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:23.737258 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:23.763584 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 109842us
W20260709 12:20:23.755300 26309 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 2.236s (SENT)
I20260709 12:20:23.726375 32226 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:23.756453 32407 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:20:23.771359 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.772066 25712 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:23.772323 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3436) took 1175 ms. Trace:
I20260709 12:20:23.755762 32401 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:23.777853 32416 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:23.772763 25708 rpcz_store.cc:276] 0709 12:20:22.596773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.596922 (+   149us) service_pool.cc:224] Handling call
0709 12:20:22.619400 (+ 22478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:23.076334 (+456934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:23.076376 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:23.131084 (+ 54708us) log.cc:844] Serialized 26 byte log entry
0709 12:20:23.360975 (+229891us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:23.361007 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:23.361093 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:23.772074 (+410981us) raft_consensus.cc:1685] finished
0709 12:20:23.772100 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:23.772255 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":217957,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6191,"thread_start_us":625397,"threads_started":2,"wal-append.queue_time_us":309428}
I20260709 12:20:23.778641 32416 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.779611 32257 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:23.745682 32271 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:23.755576 32238 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.752259 32197 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:23.773123 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:23.784613 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:23.788089 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:23.790619 32387 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:23.790954 26307 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:23.801290 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 168479us
I20260709 12:20:23.791086 32387 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:23.782939 32297 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.807771 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:23.799813 31595 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:20:23.809458 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 54205.4us
I20260709 12:20:23.813745 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:23.774667 32294 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:20:23.814378 25630 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.776841 32312 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.814883 31595 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:23.823909 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:23.826615 31595 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:23.770102 32417 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.811112 32387 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:23.830993 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:23.794766 32271 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.827956 32312 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:23.838653 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 62318.1us
I20260709 12:20:23.843556 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:23.795974 32410 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:20:23.835150 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.837438 25779 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:20:23.738401 32391 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.796164 32238 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.812263 32404 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:23.846297 25780 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:23.831780 32377 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:23.857151 31898 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:23.867046 31898 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.860420 32391 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:23.869753 31898 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:23.857271 25779 pending_rounds.cc:85] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 11
I20260709 12:20:23.870962 32404 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:23.872228 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:23.872115 32404 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.873088 32409 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:23.873479 32409 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:23.873850 32409 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:23.874650 32012 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:23.875202 32012 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.879288 32296 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.861768 32259 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:23.882421 32411 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.173s	user 0.000s	sys 0.002s
W20260709 12:20:23.886145 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44552.9us
I20260709 12:20:23.892964 32268 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:23.886449 32177 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:23.894112 32177 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:23.894510 32177 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:23.893011 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:23.902674 32297 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:23.903208 32297 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:23.904168 32297 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:23.910686 31722 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:23.914901 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:23.918015 25584 auto_rebalancer.cc:893] Removing replace marker failed: Replica 0ae979356d694a09be4b0cd976ea354a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:23.918543 25584 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:20:23.918835 25584 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:20:23.919305 32238 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.920240 25638 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:23.870970 25779 pending_rounds.cc:107] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:20:23.924242 25779 raft_consensus.cc:2889] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:23.941814 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 139682us
I20260709 12:20:23.942914 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:23.943992 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.951102 25630 raft_consensus.cc:2468] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 21.
I20260709 12:20:23.898231 32375 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:23.952067 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:23.954689 25706 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:23.856509 32416 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:23.952152 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:23.944878 25636 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:23.953194 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:23.956935 25709 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.951561 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:23.953620 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.958173 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.958324 25711 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:23.949381 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:23.960421 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:23.961706 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.962311 25630 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:23.956647 25786 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:23.878364 32012 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:23.967379 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.969880 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.970285 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:23.970512 25629 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:23.968004 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:23.976140 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.974187 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:23.967533 26308 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:23.888259 32417 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:23.969668 32423 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:23.945336 32401 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:23.991457 32401 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:23.950724 32150 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:23.979176 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:23.979918 26341 raft_consensus.cc:2368] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 20. Current term is 21.
I20260709 12:20:23.979908 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.976766 25637 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:23.982331 32422 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.982857 32396 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:23.982795 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.983657 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:23.984728 26308 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:23.964730 32259 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:23.990921 32403 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:23.954949 32375 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:23.992560 25634 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:23.993754 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.024263 32396 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:23.999872 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:23.997367 32150 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.056530 32150 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.030398 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.042570 32414 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.058821 32414 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.060010 25783 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:24.044039 32252 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:24.062273 32252 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.062965 32424 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:24.039592 32259 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.046747 32101 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.047020 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.064904 32101 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.065261 32177 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.065901 25751 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:24.066926 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.069049 32150 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:24.069540 32150 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.069972 32150 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:24.070659 32037 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:20:24.072029 32368 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:24.072434 32368 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:24.073294 32368 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:24.047734 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 67609.2us
I20260709 12:20:24.074898 32368 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:24.048180 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:24.075735 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:24.036135 32403 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:24.076680 32403 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:24.077667 31722 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:24.078104 31722 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.078430 31722 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:24.080863 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:24.084323 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 141572us
I20260709 12:20:24.066097 32422 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:24.047938 32425 raft_consensus.cc:2804] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:24.047127 32375 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.085246 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:24.085342 32425 raft_consensus.cc:697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:24.085976 32375 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.077299 31595 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.065918 32259 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.086514 32425 consensus_queue.cc:237] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.088364 31595 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.089054 31595 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:24.089076 25712 raft_consensus.cc:2393] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 18: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:24.039399 25786 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:24.111865 25709 raft_consensus.cc:2468] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:24.112630 32146 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.114799 32398 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:24.076859 32409 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.123361 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:24.123507 32252 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.125960 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:24.126626 32321 consensus_meta.cc:287] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.004s
I20260709 12:20:24.126545 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.128026 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.134374 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.136233 32250 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.136739 32226 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.136859 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.137305 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:24.041713 32396 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:24.130055 25677 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:24.144178 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.145720 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:24.148419 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.150348 25630 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:24.157029 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.159693 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.163568 26335 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:24.164390 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.164206 32226 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.165009 32012 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:24.165465 32012 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.165804 32012 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:24.167065 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 43860us
I20260709 12:20:24.168664 32387 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.169481 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:24.169783 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:24.170619 32385 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:24.171043 32385 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.171406 32385 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:24.172025 32409 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.173925 32415 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:23.995986 32423 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.149333 32130 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:24.174976 32130 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:24.175386 32130 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:24.176407 32423 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.177598 25782 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:24.180070 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.180663 25782 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
W20260709 12:20:24.181869 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51918.1us
I20260709 12:20:24.183573 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:24.194785 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.203874 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.204411 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:24.207407 25635 raft_consensus.cc:2393] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 14: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:24.205471 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.204063 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:24.142654 32416 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:20:24.219769 26339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a760df354a14284b004635d45d88a3e"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:24.221873 26339 raft_consensus.cc:606] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.222342 26339 raft_consensus.cc:614] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:24.184459 32226 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.204774 32423 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:20:24.194252 32391 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:20:24.182072 32399 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:24.226410 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:24.193706 32394 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:24.228045 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.142215 32150 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.147256 31722 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.143399 32146 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.152141 32252 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.072s
W20260709 12:20:24.244257 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74544.1us
I20260709 12:20:24.152602 32321 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.219821 32037 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:24.254580 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:24.256235 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.258827 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:24.259253 32383 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:24.259745 25781 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:24.181449 32250 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.259747 25630 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:24.263656 25677 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:24.265062 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.264966 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:24.178300 32398 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.231967 32394 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.271008 32377 raft_consensus.cc:2804] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:24.274634 25750 leader_election.cc:400] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 20. Current term is 21.
I20260709 12:20:24.277894 25750 leader_election.cc:403] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:24.261036 32037 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.265527 32294 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:20:24.315949 26309 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.279891 32394 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:24.318501 25750 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:24.319283 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.283357 32421 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:24.323153 32177 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.318925 32270 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:24.323995 32428 log.cc:927] Time spent T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:20:24.324048 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.326838 32431 tablet_replica.cc:333] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:20:24.327647 32427 log.cc:927] Time spent T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:20:24.318676 26309 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.987s (SENT) [suppressed 22 similar messages]
I20260709 12:20:24.315903 32409 raft_consensus.cc:3060] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.336531 31722 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.307898 32259 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:20:24.316314 32435 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:24.340109 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.340896 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.444109 25711 raft_consensus.cc:2410] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:20:24.444722 25781 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:24.445253 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.446058 25711 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:20:24.426720 32385 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.447391 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:24.447974 32385 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.327656 32321 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.366734 32421 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:24.344342 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.451800 25707 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:24.422719 32388 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:24.278395 32377 raft_consensus.cc:697] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:24.348039 25708 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:20:24.374379 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:24.456804 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:24.379823 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 61425.1us
I20260709 12:20:24.312747 32037 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:24.433842 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.341550 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.402176 32226 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:24.343530 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.345520 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.460987 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:24.468204 25713 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:20:24.470031 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:24.470743 25634 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:24.452605 32296 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:24.431526 32270 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.477828 25707 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:24.480258 32377 consensus_queue.cc:237] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.481524 32435 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.482687 32321 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:24.483438 32321 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.485283 32418 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:24.490077 32270 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.491447 32396 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.493551 32396 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.499157 25708 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:24.500393 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.501715 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.504575 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.508373 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.508613 32150 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.509887 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.510178 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.463979 32312 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:24.506295 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.515647 25705 raft_consensus.cc:2410] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:20:24.502363 25708 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:24.517019 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.520143 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.517899 25704 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:24.485904 32418 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.524966 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.486274 32321 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.527053 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.482075 32435 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.527072 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.470590 32388 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.528196 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.514657 32101 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.534509 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.527590 32421 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.551010 32421 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:24.551184 32401 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.552511 32387 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.553400 32403 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.525207 32296 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:24.555531 32296 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.556794 32250 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.557479 32398 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.559648 32418 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.568830 25634 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:24.571130 25706 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:24.535975 32435 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.536667 25713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:50646
W20260709 12:20:24.533792 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 215124us
W20260709 12:20:24.544052 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 104436us
I20260709 12:20:24.535107 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.579491 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.580588 25704 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:24.581780 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.583159 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.542948 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:24.579553 25707 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:24.583963 26343 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 11.
I20260709 12:20:24.589037 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.590256 25708 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:24.584820 26335 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:24.597080 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.597236 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.597782 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.598310 26341 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:24.601652 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.602205 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.604724 26344 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 20: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:24.598443 26337 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:24.603739 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.584077 32380 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.592849 32417 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.628823 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:24.629757 25637 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:24.635217 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.640684 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.604256 32398 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:24.687665 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.647575 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.650414 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:24.689245 26344 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:24.689455 25781 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:24.689769 25638 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:24.645618 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.691355 25636 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:24.691128 32417 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.637084 32012 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.693334 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.693557 32417 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.693912 25782 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.619102 32414 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.654567 32130 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.686170 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:24.696661 32130 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:24.657410 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 114609us
I20260709 12:20:24.659021 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.697459 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.660606 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:24.678706 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:24.685197 26307 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.700219 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.700398 25676 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:24.701119 25781 raft_consensus.cc:2393] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:24.701506 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.701639 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:24.702111 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.702358 26340 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:24.645437 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:24.637161 32380 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.703691 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.704422 25779 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:24.705214 32396 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:24.705520 32380 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.694736 25782 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:24.705932 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.708593 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.694764 32012 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.708521 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:24.705848 32396 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:24.685170 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.725675 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:24.712391 25704 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:24.714319 25635 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
W20260709 12:20:24.718823 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 73499.9us
I20260709 12:20:24.709369 25782 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
W20260709 12:20:24.723179 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.636090 32297 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:24.734549 26337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:24.735400 26337 raft_consensus.cc:606] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:24.649065 32435 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.738785 25706 raft_consensus.cc:1240] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:24.721405 32402 log.cc:927] Time spent T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:20:24.696955 32259 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:24.751792 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.748642 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:24.749507 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.755594 25782 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:24.732820 32441 log.cc:927] Time spent T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.154s	user 0.002s	sys 0.000s
I20260709 12:20:24.747437 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:24.768194 25603 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:24.740463 31595 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:20:24.773972 25713 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:24.774590 25713 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:24.746915 32304 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:24.778205 26344 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:24.783761 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.785154 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.734855 32130 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:20:24.785988 25778 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 20: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:24.787199 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.782588 26340 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:24.742146 32270 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:20:24.787608 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.766078 32268 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:24.791308 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.791679 25781 raft_consensus.cc:1240] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:24.775710 32238 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:24.784535 32409 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 20. Current term is 21.
I20260709 12:20:24.796793 32294 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:24.806648 32294 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:24.798919 32403 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.807941 32312 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:24.808534 32312 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.810691 32414 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:24.812057 25785 raft_consensus.cc:1240] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:24.776360 32259 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:24.804843 25787 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:20:24.795374 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:24.815263 32146 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.817835 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:24.831545 25780 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:24.818888 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:24.835742 25632 raft_consensus.cc:2368] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18.
I20260709 12:20:24.837954 25786 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:24.821396 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:24.827133 32271 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:24.827596 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:24.828039 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.840577 26340 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:24.828933 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.840867 25779 raft_consensus.cc:2393] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:24.829139 25752 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:24.841361 25710 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:24.846011 25782 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:24.850386 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.853287 32431 raft_consensus.cc:2243] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.853677 25706 raft_consensus.cc:1240] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:24.854163 32312 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.854589 32431 raft_consensus.cc:2272] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.842145 31595 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:24.855824 31781 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:20:24.863731 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.866717 32431 ts_tablet_manager.cc:1916] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:24.776093 32435 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.803274 32238 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:24.872278 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.812558 32448 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.877130 25637 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:24.813023 32250 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:24.881982 32443 raft_consensus.cc:2764] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:24.882153 32250 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:24.885084 32451 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:20:24.886039 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:24.877847 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.878883 32146 raft_consensus.cc:2804] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:20:24.888726 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 70975.8us
I20260709 12:20:24.889022 32146 raft_consensus.cc:697] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:24.889611 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:24.890223 25713 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.891360 25636 raft_consensus.cc:1275] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:24.894996 25676 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:24.878634 25780 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:24.896857 32448 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.903679 32448 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.832262 32037 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.904556 32395 consensus_queue.cc:1059] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:24.907224 32387 raft_consensus.cc:3055] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:20:24.907600 32394 consensus_meta.cc:287] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 12:20:24.907678 32387 raft_consensus.cc:740] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:24.890054 32146 consensus_queue.cc:237] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.908179 32394 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.908864 32387 consensus_queue.cc:260] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.910920 25707 raft_consensus.cc:2368] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:20:24.910286 32387 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.912436 32435 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.828250 32396 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:24.915961 32197 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:24.917299 32394 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:24.917732 32197 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:24.870316 32420 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:24.878736 32238 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:24.920483 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.870122 32409 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:24.920917 25710 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:24.886207 32250 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:24.921073 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.941033 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.941341 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.873605 32388 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:24.922886 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:24.923167 32259 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:24.925743 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:24.926592 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 56365.5us
I20260709 12:20:24.897401 32312 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:24.920919 32401 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:24.937558 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:24.949304 25780 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:24.939954 26340 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:24.942281 25637 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:24.932964 25676 raft_consensus.cc:2686] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:24.942041 25786 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:24.945897 26307 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:24.965860 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 70975.9us
I20260709 12:20:24.967303 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.966821 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:24.966754 26307 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:24.973251 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.974255 25782 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:24.976126 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.980237 25780 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:24.982240 32394 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:24.984381 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.990887 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:24.991577 25778 raft_consensus.cc:3060] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:24.992331 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.993013 25711 raft_consensus.cc:2376] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16. Re-sending same reply.
I20260709 12:20:24.997545 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:24.999140 25637 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:24.999673 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.000756 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.001049 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:25.001139 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.001672 26339 raft_consensus.cc:2468] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:25.002880 25710 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:20:25.003202 25677 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:25.005664 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:25.005856 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 211879us
I20260709 12:20:25.007352 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:25.009210 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:25.013926 26342 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:25.010320 25752 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.015777 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:25.017139 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.017895 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:24.944290 32420 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.022567 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:25.025947 25708 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:24.948798 32439 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:20:25.028908 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.029086 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:25.029554 26337 raft_consensus.cc:1813] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Handling vote request from an unknown peer c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.030526 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:24.978866 32405 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.020995 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.019148 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:25.027568 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:24.970182 32197 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:25.019459 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.040337 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:24.998529 32401 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.954715 32409 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.037930 26337 raft_consensus.cc:2368] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18.
I20260709 12:20:24.997975 32403 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:25.053303 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.028581 31595 raft_consensus.cc:2764] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:20:25.028591 32448 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:25.068892 25752 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.047325 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.058125 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:25.060071 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:25.044234 32405 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.071537 25752 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:25.062450 32409 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:25.058462 32403 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:25.063202 32445 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:25.035888 32130 raft_consensus.cc:2804] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:25.054205 32401 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:25.056746 26309 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Timed out: RequestConsensusVote RPC to 127.23.217.66:39947 timed out after 1.523s (SENT)
I20260709 12:20:25.091135 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.074447 25752 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.050752 32422 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:25.096786 26335 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:25.072398 32101 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:25.103979 32130 raft_consensus.cc:697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:25.089426 25778 raft_consensus.cc:2468] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
W20260709 12:20:25.092900 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 93760.4us
I20260709 12:20:25.070946 32250 consensus_queue.cc:1059] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:20:25.078393 32146 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.115082 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:25.115383 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 58660.7us
I20260709 12:20:25.116238 26340 raft_consensus.cc:606] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:25.084986 32421 consensus_meta.cc:287] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.002s
I20260709 12:20:25.116678 26340 raft_consensus.cc:614] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:25.082607 32312 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:25.117298 26342 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:25.132704 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:25.132076 32462 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:25.105921 32448 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:25.154404 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:25.079229 32435 consensus_queue.cc:1059] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:20:25.178331 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.165001 32312 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:25.172833 32037 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.108390 32130 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.201767 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:25.202461 25778 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:25.198931 32431 ts_tablet_manager.cc:1929] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.10
I20260709 12:20:25.208695 32390 raft_consensus.cc:3055] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:25.208662 32431 log.cc:1199] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/ef4fd24cfa184aa8a8a1de8f32296e6a
I20260709 12:20:25.209245 32390 raft_consensus.cc:740] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:25.210860 32431 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35600 (request call id 0) took 1163 ms. Trace:
I20260709 12:20:25.195860 32405 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.175491 32197 consensus_queue.cc:1059] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:20:25.211388 32431 rpcz_store.cc:276] 0709 12:20:24.047536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:24.047813 (+   277us) service_pool.cc:224] Handling call
0709 12:20:24.275454 (+227641us) ts_tablet_manager.cc:1151] Deleting tablet ef4fd24cfa184aa8a8a1de8f32296e6a
0709 12:20:24.312221 (+ 36767us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:20:25.109771 (+797550us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:25.198758 (+ 88987us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:25.210740 (+ 11982us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175624,"tablet-delete.queue_time_us":177573,"thread_start_us":106393,"threads_started":1}
I20260709 12:20:25.192270 32466 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:25.213788 26308 leader_election.cc:400] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18.
I20260709 12:20:25.214088 26308 leader_election.cc:403] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:20:25.215574 32462 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18.
I20260709 12:20:25.216492 32462 consensus_queue.cc:1048] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:25.218978 25629 raft_consensus.cc:1240] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:25.219965 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:25.228731 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 62045.4us
I20260709 12:20:25.192515 32465 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:25.232699 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:25.233614 26340 raft_consensus.cc:606] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
W20260709 12:20:25.234205 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60527.4us
W20260709 12:20:25.234871 32470 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:20:25.235584 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:25.236765 32468 log.cc:927] Time spent T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:20:25.230500 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:25.234210 26340 raft_consensus.cc:614] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:25.246037 32401 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.246281 32466 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:25.192832 32403 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.246644 25752 leader_election.cc:400] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:20:25.246698 32466 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.247284 25752 leader_election.cc:403] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:20:25.247594 32238 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.249184 25630 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 11. Current term is 12. Ops: []
W20260709 12:20:25.233523 32467 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:20:25.249694 32238 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:25.251149 32466 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:25.255613 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:25.317348 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 70877.9us
I20260709 12:20:25.316995 32396 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:20:25.319393 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66644.4us
I20260709 12:20:25.324560 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:25.325313 25786 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:25.321874 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:25.326507 26339 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 12:20:25.327396 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 72004us
W20260709 12:20:25.329377 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 78734.9us
W20260709 12:20:25.333690 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 98156.5us
I20260709 12:20:25.333927 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:25.334722 25707 raft_consensus.cc:2376] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14. Re-sending same reply.
I20260709 12:20:25.335227 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.319732 32444 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:25.352065 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:25.354998 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:25.323100 32448 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:20:25.361658 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:25.316918 32405 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:20:25.359133 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.254024 32380 consensus_queue.cc:1059] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:20:25.366981 32444 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.373512 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.332208 32377 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:25.344856 32420 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:25.353560 32259 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.379026 32095 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.233840 32312 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.408221 32444 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:25.414780 32387 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:25.393766 32259 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.416432 32398 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:25.393267 32095 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.418850 32095 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:25.429291 32479 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:25.433176 25713 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:25.418141 32259 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.417047 32398 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:25.434053 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.447060 25601 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.458657 32252 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:25.459090 32252 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.459426 32252 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:25.459707 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:25.460050 32408 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:25.460314 25637 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:25.460716 32408 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:25.463022 25636 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:20:25.464983 32469 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:25.466764 32294 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.467880 32294 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:25.469314 32101 raft_consensus.cc:3055] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:25.469746 32101 raft_consensus.cc:740] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:25.469806 32252 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:25.480710 32476 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 12:20:25.481868 32460 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.451s	user 0.002s	sys 0.000s
I20260709 12:20:25.456940 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:25.489785 32368 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:25.490413 32368 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:25.491411 32368 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.492486 32466 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.462046 32408 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.496909 32398 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.456877 32399 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:25.500209 32445 raft_consensus.cc:3055] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:25.501020 32445 raft_consensus.cc:740] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:25.500874 32399 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.503293 32399 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.496909 32101 consensus_queue.cc:260] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.503233 32465 raft_consensus.cc:3055] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:25.506985 32465 raft_consensus.cc:740] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:25.508284 32465 consensus_queue.cc:260] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.502360 32445 consensus_queue.cc:260] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.463999 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet ef4fd24cfa184aa8a8a1de8f32296e6a (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:25.538046 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:25.515939 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.558012 32424 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:25.538544 25522 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 98305.5us
W20260709 12:20:25.597481 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.524704 25713 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:25.591876 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:25.594142 25712 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
W20260709 12:20:25.606451 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.587894 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.558197 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:25.588356 25631 raft_consensus.cc:1240] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:25.599193 25637 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
W20260709 12:20:25.559263 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 43795.7us
I20260709 12:20:25.612794 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.616616 25782 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:25.627398 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:25.630539 25708 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:25.587159 32101 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:25.631529 25780 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:20:25.619071 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:25.635356 32394 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:25.635942 32394 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.638022 32394 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.584040 32398 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:25.639431 32394 consensus_queue.cc:1059] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:25.640333 32399 raft_consensus.cc:3055] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:25.640738 32399 raft_consensus.cc:740] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:25.641534 32250 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:25.583199 32252 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.531073 32390 consensus_queue.cc:260] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.544217 31595 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:25.645167 31595 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.645887 32409 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:25.646363 32409 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.641661 32399 consensus_queue.cc:260] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:25.647816 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 158616us
I20260709 12:20:25.658185 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.659070 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:25.496465 32377 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:25.562084 32465 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:25.582446 32445 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:25.663546 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 290083us
I20260709 12:20:25.576829 32466 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:25.677839 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.682475 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:25.696583 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:25.724736 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 59840.7us
I20260709 12:20:25.725100 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.649595 32250 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.649071 32409 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:25.649228 32304 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:25.731911 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50321.5us
I20260709 12:20:25.663295 32444 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:25.657191 32197 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:25.668437 32268 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:25.738432 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.756577 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.761016 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.736785 32271 raft_consensus.cc:2764] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:20:25.771036 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.771772 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.732581 32396 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:25.739851 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.739341 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:25.768765 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 403158us
I20260709 12:20:25.784679 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:25.785575 25635 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
W20260709 12:20:25.789501 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 108405us
I20260709 12:20:25.790304 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.741148 32399 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:25.792927 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 53715.6us
I20260709 12:20:25.795292 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.754782 32444 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.799708 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.760541 32481 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:25.757930 32095 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:20:25.804965 25677 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:25.762014 32197 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.812728 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:25.737073 32250 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:25.816368 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:25.822003 25778 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:25.813529 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:25.797084 32144 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:25.804406 32311 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:25.817857 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:25.826045 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:25.826892 25778 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:25.828440 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:25.831327 32390 raft_consensus.cc:3060] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:25.833149 32294 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:25.833762 32294 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:25.836005 32294 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:25.836426 25778 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
W20260709 12:20:25.840317 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:25.940949 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 93700.1us
I20260709 12:20:25.937134 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:25.938110 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 124706us
I20260709 12:20:25.800809 32480 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:20:25.986282 25784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.015s	sys 0.023s
W20260709 12:20:26.006240 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 177979us
W20260709 12:20:26.006744 25636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.010s	sys 0.030s
I20260709 12:20:25.802054 32259 consensus_queue.cc:1059] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:20:25.817049 32383 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:25.825248 32144 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:26.018831 25637 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:20:25.820956 32481 raft_consensus.cc:2764] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:26.052215 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.057636 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:26.058540 25780 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:26.044203 26307 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:25.982542 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:25.988166 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.932773 32420 raft_consensus.cc:2764] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:20:26.038615 25784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.019s	sys 0.024s
W20260709 12:20:26.063365 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 330628us
I20260709 12:20:25.837971 31595 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:25.933748 32304 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:26.041565 25636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.011s	sys 0.031s
W20260709 12:20:26.010902 32470 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 12:20:25.934131 32294 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:26.060868 25780 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:20:26.017695 32252 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:26.064816 25782 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:26.021420 26335 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:26.062330 26339 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:26.070370 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:26.053674 32408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.019s	sys 0.032s
I20260709 12:20:26.043103 32271 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:26.051292 32469 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:26.091619 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:26.055124 32144 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:26.076361 32423 raft_consensus.cc:2764] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:20:26.115437 32482 log.cc:927] Time spent T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:20:26.108929 32271 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.090791 32294 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:26.098546 32408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.019s	sys 0.033s
I20260709 12:20:26.084132 32420 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:26.110641 32469 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.118424 32101 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:26.138775 32439 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.139453 32439 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.141661 32439 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:26.146939 26336 raft_consensus.cc:1275] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:26.147051 32444 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:26.118804 32485 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:20:26.149801 32398 consensus_meta.cc:287] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.511s	user 0.005s	sys 0.005s
I20260709 12:20:26.119063 32484 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:26.151196 32484 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.151298 32250 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:26.151762 32484 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:26.121766 25633 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:26.150395 25704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3482) took 1141 ms. Trace:
I20260709 12:20:26.158414 25704 rpcz_store.cc:276] 0709 12:20:25.009036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.032723 (+ 23687us) service_pool.cc:224] Handling call
0709 12:20:25.053102 (+ 20379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:25.182343 (+129241us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:20:25.182378 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:25.226281 (+ 43903us) log.cc:844] Serialized 26 byte log entry
0709 12:20:25.648431 (+422150us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:25.648471 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:25.648547 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:26.150136 (+501589us) raft_consensus.cc:1685] finished
0709 12:20:26.150167 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:26.150315 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12585,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":60,"thread_start_us":521054,"threads_started":2,"wal-append.queue_time_us":551159}
I20260709 12:20:26.138296 32469 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:26.187217 32398 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.160846 32480 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:20:26.194805 32480 consensus_queue.cc:1243] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer c7bf8f20701245bcbaa09cc663025f97 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:26.194630 32398 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.197007 32398 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:26.187530 32380 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:26.198009 32380 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:26.188367 32487 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:20:26.200212 32380 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.122921 32420 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.203289 32420 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:26.189533 31595 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:26.222319 31595 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.223625 32409 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.224778 31595 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.226342 25779 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 15: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:26.227584 32409 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:26.238268 32484 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.240829 32484 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:26.249372 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:26.249948 32144 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.250321 32409 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:26.251583 25751 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:26.252317 32409 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:26.253392 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:26.253041 25629 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:26.257306 25778 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:20:26.269791 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:26.282369 25782 raft_consensus.cc:2393] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 21: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:20:26.257623 32465 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.001s
I20260709 12:20:26.292908 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:26.293814 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:26.328135 32492 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:20:26.330291 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1315) took 1228 ms. Trace:
I20260709 12:20:26.330859 25784 rpcz_store.cc:276] 0709 12:20:25.101730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.101897 (+   167us) service_pool.cc:224] Handling call
0709 12:20:25.115274 (+ 13377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:25.118173 (+  2899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:25.118205 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:25.156153 (+ 37948us) log.cc:844] Serialized 26 byte log entry
0709 12:20:25.278123 (+121970us) spinlock_profiling.cc:243] Waited 9.6 ms on lock 0x7b18004a41c0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b2884e07 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b28a1c02 00007f29b552920b 00007f29b552ed23 00007f29b552f0d1 00007f29b594aa0f
0709 12:20:26.063871 (+785748us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:26.063907 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.063991 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:26.329875 (+265884us) raft_consensus.cc:1685] finished
0709 12:20:26.329903 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:26.330216 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3764,"raft.queue_time_us":2619,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":31606,"spinlock_wait_cycles":22078208,"thread_start_us":740436,"threads_started":1,"wal-append.queue_time_us":1054421}
I20260709 12:20:26.296834 32420 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:26.335124 32420 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.338021 32420 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:26.339267 32271 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:26.290416 32294 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:26.339442 32491 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:20:26.349718 25778 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 14: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:26.348783 25783 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
W20260709 12:20:26.266862 32037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.004s	sys 0.031s
I20260709 12:20:26.284929 32409 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:26.358574 32037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.007s	sys 0.031s
I20260709 12:20:26.367296 25750 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:26.361099 32409 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:26.380219 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 313753us
W20260709 12:20:26.376169 26343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56156 (request call id 1236) took 1374 ms (client timeout 1766 ms). Trace:
W20260709 12:20:26.376659 32445 consensus_meta.cc:287] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.000s
W20260709 12:20:26.389788 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:26.389931 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.393129 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:26.368103 32420 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.396275 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.396544 32502 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:26.396765 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.393787 32383 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:26.397549 25632 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:26.398128 32383 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.399165 32499 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:26.388705 26343 rpcz_store.cc:269] 0709 12:20:25.000075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.000855 (+   780us) service_pool.cc:224] Handling call
0709 12:20:26.372403 (+1371548us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54004975d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:26.373954 (+  1551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144365824}
W20260709 12:20:26.393918 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 130049us
W20260709 12:20:26.391453 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:26.374631 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 291647us
I20260709 12:20:26.400818 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:26.387521 32271 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:26.387081 32409 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:26.401959 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.402052 32502 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:26.401106 32420 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.402632 32507 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:20:26.406399 32144 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.405128 32271 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.428851 32197 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:26.418839 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.405489 25637 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:26.429266 32271 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:26.405601 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 361494us
I20260709 12:20:26.404911 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:26.413056 32502 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:26.443166 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.423233 32499 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.443917 25631 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:26.445017 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.446771 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.446959 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:26.447192 32420 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.447850 25638 raft_consensus.cc:2393] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 15: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:26.448011 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.448644 32499 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:26.449745 32472 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:26.450053 32472 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.450276 32472 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.451898 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.441879 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.429294 32503 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:20:26.455236 25710 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:26.456306 32472 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.441907 25676 leader_election.cc:400] T ef4fd24cfa184aa8a8a1de8f32296e6a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18.
I20260709 12:20:26.458472 32472 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:26.458313 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:26.446237 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.460799 25631 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:26.463522 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:26.466888 25709 raft_consensus.cc:1240] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:26.473093 32508 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:26.476037 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.477686 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.478900 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.480073 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:26.482347 26342 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:20:26.483831 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.483991 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:26.484862 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:26.484970 25711 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:26.485559 25778 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.483918 32461 consensus_queue.cc:1059] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:20:26.489667 32461 raft_consensus.cc:3055] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:26.489538 32271 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.490166 32461 raft_consensus.cc:740] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:26.491178 32461 consensus_queue.cc:260] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.492421 32461 raft_consensus.cc:3060] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.474385 32465 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:26.483836 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:26.499444 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.467042 32472 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:26.481518 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:26.500946 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:26.500880 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.503139 25709 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:26.503392 26343 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:26.501469 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:26.506536 32481 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:26.506946 32481 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:26.507697 32481 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.508261 32252 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:26.508945 32481 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:26.510742 25711 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:26.515923 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:26.517310 32420 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:26.517836 32420 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.518280 32420 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:26.521307 26307 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:26.526309 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:26.527863 25708 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:20:26.528209 25708 pending_rounds.cc:85] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 14
I20260709 12:20:26.528431 25708 pending_rounds.cc:107] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:20:26.528650 25708 raft_consensus.cc:2889] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:26.529531 32473 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:26.529958 32473 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.530305 32473 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:26.520644 32505 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:26.531057 32505 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.531412 32505 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:26.536384 25636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2793) took 1062 ms. Trace:
I20260709 12:20:26.536926 25636 rpcz_store.cc:276] 0709 12:20:25.473849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.474010 (+   161us) service_pool.cc:224] Handling call
0709 12:20:25.474369 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:25.474602 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:25.474631 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:25.475218 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:20:26.075972 (+600754us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:26.076007 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.076076 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:26.530515 (+454439us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b2400121b08. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27289c4 00007f29b2726acb 00007f29b28e1dc9 00007f29b28e1a55 00007f29b28e0fb0 00007f29b5557488 00007f29b5557be8 00007f29b552f398 00007f29b552a76a 00007f29b5529bb6
0709 12:20:26.535980 (+  5465us) raft_consensus.cc:1685] finished
0709 12:20:26.536011 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:26.536277 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133699328,"thread_start_us":600395,"threads_started":1,"wal-append.queue_time_us":641973}
I20260709 12:20:26.538854 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:26.539613 25783 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 21.
I20260709 12:20:26.539922 25784 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:26.540925 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:26.544169 32464 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:26.545017 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:26.546176 25779 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:26.544974 25636 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:26.473649 32508 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.555496 32464 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:26.556587 32464 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.558951 32271 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.503446 25778 raft_consensus.cc:2468] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 22.
I20260709 12:20:26.560696 32508 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.561064 25676 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:26.562685 32472 raft_consensus.cc:2804] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.563407 32472 raft_consensus.cc:697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:26.564661 32472 consensus_queue.cc:237] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.566100 25709 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:26.575067 25750 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:26.576656 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.577798 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.578555 26342 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:26.581056 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:26.569109 32465 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:26.485953 32499 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:26.577292 26336 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:26.589404 25713 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:26.593302 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:26.594066 26336 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:26.596860 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:26.590167 32445 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:26.614374 32466 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:26.616603 32390 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:26.590003 32403 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
W20260709 12:20:26.642724 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 43879.6us
I20260709 12:20:26.644018 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:26.646080 26335 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:26.646700 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:26.647926 26338 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:26.614694 32439 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:20:26.628533 32445 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:26.612936 32499 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.636793 32466 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.617914 32500 consensus_queue.cc:1048] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:20:26.671053 32473 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.663461 32445 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:26.652921 32271 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:26.685371 32271 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.685843 32271 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:26.696458 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.693171 32250 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.698740 32250 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.700982 32250 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.665328 32466 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:26.710685 25679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.015s	sys 0.039s
W20260709 12:20:26.711165 25679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.016s	sys 0.040s
I20260709 12:20:26.714649 26335 raft_consensus.cc:3060] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:26.724416 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 88428.8us
W20260709 12:20:26.732234 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.734931 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.750897 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:26.751938 25704 raft_consensus.cc:2410] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:20:26.752758 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.753851 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.757050 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.766211 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.778460 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2786) took 1340 ms. Trace:
I20260709 12:20:26.779073 25633 rpcz_store.cc:276] 0709 12:20:25.438403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.438694 (+   291us) service_pool.cc:224] Handling call
0709 12:20:25.464605 (+ 25911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:26.152491 (+687886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.778176 (+625685us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54001c3ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:26.778384 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510856960}
I20260709 12:20:26.785491 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:26.788045 25709 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:26.791833 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.794396 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:26.797300 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:26.798096 25711 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:26.801949 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:26.803130 25713 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:26.826257 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 112821us
I20260709 12:20:26.816197 32500 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.821604 32294 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.838035 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.861930 25706 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:26.849268 32294 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:26.869792 32409 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.871047 32294 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:26.890590 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.901690 26336 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:26.830783 32271 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.841521 32500 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:26.916445 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:26.920125 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 496127us
W20260709 12:20:26.912007 32514 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:20:26.957309 26340 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:26.971477 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.964668 25677 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:26.860802 25713 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:26.979005 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.958431 32294 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.964166 32499 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:26.972620 32409 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.006978 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:26.977336 32394 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:27.006649 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:26.984351 32465 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.983798 32377 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.984054 32472 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.027838 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:26.971328 32445 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.033726 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:27.034103 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 159613us
I20260709 12:20:27.038774 25713 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
W20260709 12:20:27.011276 32481 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.502s	user 0.005s	sys 0.009s
I20260709 12:20:26.977200 32144 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:20:27.055492 32252 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:27.056775 25752 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.051951 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:27.059468 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 52965.4us
I20260709 12:20:26.995656 32294 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.015775 32394 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:27.061269 32485 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:20:27.069883 32445 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.071099 26344 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:27.075268 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:27.079986 25752 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.049981 32472 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.088552 25601 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.001089 32420 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.128571 25712 pending_rounds.cc:85] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 10
I20260709 12:20:27.145859 25712 pending_rounds.cc:107] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:20:27.146394 25712 raft_consensus.cc:2889] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.089874 32394 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:27.035845 32465 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.049671 32377 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:27.183522 32522 log.cc:927] Time spent T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 12:20:27.183871 32532 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:20:27.107957 32496 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
W20260709 12:20:27.185506 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 110609us
I20260709 12:20:27.169991 32529 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:27.173605 32416 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:20:27.208577 32473 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.209151 32545 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:27.212101 32377 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:27.214864 32514 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.129s	user 0.002s	sys 0.000s
I20260709 12:20:27.215492 32420 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:27.215857 32294 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.202519 32499 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:27.216404 32271 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.219182 32294 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:27.220044 32294 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.221179 32466 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.222450 26337 raft_consensus.cc:2368] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 21. Current term is 22.
I20260709 12:20:27.222574 32294 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.227098 32500 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.185431 32547 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:27.198598 32503 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:27.196447 32538 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:27.204077 32541 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.204598 32416 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:27.238179 32528 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:27.200032 32529 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:27.255637 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 175707us
I20260709 12:20:27.278527 25786 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 12:20:27.283215 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 194732us
I20260709 12:20:27.284318 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:27.273393 32505 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.004s
I20260709 12:20:27.269981 32541 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.288163 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.289120 25785 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:20:27.289155 25783 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:27.291720 32416 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.301504 25784 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:27.302350 32529 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:27.303954 32466 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.309880 25677 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:27.310770 25601 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.326388 25752 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:27.340879 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:27.315991 32559 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:27.324762 32311 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:27.324221 32547 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:27.357973 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:27.358439 25711 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:20:27.308606 32505 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.337854 32250 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:27.369366 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.372792 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.373386 25709 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:27.373993 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:27.373286 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.333410 32445 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:20:27.356063 32556 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:27.359992 32465 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.376814 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:27.381001 25629 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 17: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:27.364102 32550 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:27.375665 32538 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:27.390295 32538 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.393447 32538 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.368228 32144 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:27.394289 32144 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.397727 32144 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.403832 32541 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.385296 32394 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.385468 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.417188 25704 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 12.
I20260709 12:20:27.352867 32461 consensus_queue.cc:1059] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:20:27.367389 32545 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:27.414701 25707 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:27.366364 32560 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:27.378309 32559 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:20:27.418336 32535 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.248s	user 0.002s	sys 0.000s
I20260709 12:20:27.377601 32549 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.416272 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.433981 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.434980 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:27.364562 32547 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.376238 32250 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.436291 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:27.440187 26342 raft_consensus.cc:2368] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17.
I20260709 12:20:27.438707 32549 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.426761 32545 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.434926 32560 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.434789 32530 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:27.456954 25535 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 300) took 45993 ms (client timeout 15000 ms). Trace:
W20260709 12:20:27.488634 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 72452.5us
I20260709 12:20:27.390048 32550 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.504818 32550 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.508250 32560 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:27.461570 32547 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:27.512763 25530 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:20:27.452894 32553 log.cc:927] Time spent T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:20:27.514680 32520 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:27.520531 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:27.528378 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:27.536433 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 163367us
I20260709 12:20:27.449838 25707 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:27.537346 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.522239 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.522221 32037 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:27.522650 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.523047 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.523339 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:27.523741 32502 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:27.524030 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.524029 32544 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:27.525256 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.547096 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.547448 25635 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 19: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:27.547760 25784 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:27.526790 26338 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:27.548429 26338 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:27.548758 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.545138 25631 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:27.527419 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.528371 32403 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:27.528671 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:27.529018 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:27.530225 32549 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.459491 32394 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.464116 32250 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.458163 32505 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.533293 32545 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.533413 32489 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:27.501335 25535 rpcz_store.cc:269] 0709 12:19:41.463468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.463739 (+   271us) service_pool.cc:224] Handling call
0709 12:20:27.456863 (+45993124us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:24.041713 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:24.127734 (+ 86021us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:24.127798 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:20:24.127818 (+    20us) tablet.cc:662] Decoding operations
0709 12:20:24.131307 (+  3489us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:24.131429 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:24.131465 (+    36us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:20:24.132553 (+  1088us) tablet.cc:701] Row locks acquired
0709 12:20:24.132606 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:20:24.132788 (+   182us) write_op.cc:270] Start()
0709 12:20:24.132955 (+   167us) write_op.cc:276] Timestamp: P: 1783599624132728 usec, L: 0
0709 12:20:24.132992 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:20:24.213421 (+ 80429us) log.cc:844] Serialized 4461 byte log entry
0709 12:20:24.703029 (+489608us) op_driver.cc:464] REPLICATION: finished
0709 12:20:24.820756 (+117727us) write_op.cc:301] APPLY: starting
0709 12:20:24.820880 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:24.915049 (+ 94169us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:24.915171 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:24.928827 (+ 13656us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:24.929255 (+   428us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 12:20:24.929314 (+    59us) write_op.cc:312] APPLY: finished
0709 12:20:24.929805 (+   491us) log.cc:844] Serialized 105 byte log entry
0709 12:20:24.949763 (+ 19958us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:24.957554 (+  7791us) write_op.cc:454] Released schema lock
0709 12:20:24.975778 (+ 18224us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:25.059984 (+ 84206us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:25.060020 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4751,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":117376,"apply.run_cpu_time_us":17480,"apply.run_wall_time_us":224022,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50173,"num_ops":11,"prepare.queue_time_us":100192,"prepare.run_cpu_time_us":18601,"prepare.run_wall_time_us":432866,"raft.queue_time_us":165744,"raft.run_cpu_time_us":20895,"raft.run_wall_time_us":63903,"replication_time_us":569872,"thread_start_us":416821,"threads_started":4,"wal-append.queue_time_us":303561,"wal-append.run_cpu_time_us":16180,"wal-append.run_wall_time_us":1405060}]]}
I20260709 12:20:27.501277 32530 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.521772 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:27.542852 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.585870 25630 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:27.549362 25783 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:27.553462 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.549294 26338 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.554737 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.555662 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:27.587513 26336 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:27.555557 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.590655 32569 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:27.558079 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:27.591236 32569 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.591660 25708 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:20:27.559867 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:27.575667 25676 leader_election.cc:400] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 21. Current term is 22.
I20260709 12:20:27.594764 25676 leader_election.cc:403] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:27.594643 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.595065 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.595376 25778 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:27.607896 26338 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:27.608114 32569 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.613408 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.617761 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:27.620447 25778 raft_consensus.cc:2368] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:20:27.618175 25782 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:27.575130 32461 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:27.599339 32530 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:27.609489 32565 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:20:27.626601 25633 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:27.631194 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.632714 25708 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 22.
I20260709 12:20:27.632942 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.635767 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.637588 26336 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:27.637887 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.632397 25785 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:27.639642 25635 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:27.640519 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.641677 25631 raft_consensus.cc:1240] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:27.642755 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.645360 25630 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:27.587726 32443 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:20:27.586572 32037 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.625994 32495 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:27.656539 25631 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:20:27.652420 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.641217 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:27.658301 25633 raft_consensus.cc:2410] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:27.657512 25636 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:27.555910 32502 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.665360 25633 raft_consensus.cc:1275] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:20:27.622946 32505 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:20:27.665741 25633 pending_rounds.cc:85] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 11
I20260709 12:20:27.668263 25633 pending_rounds.cc:107] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:20:27.668562 25633 raft_consensus.cc:2889] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.607538 32489 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.676429 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.675377 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.679190 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.665038 32461 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.666839 32416 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.654044 32095 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:20:27.682825 32461 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.685503 32416 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.681773 32575 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:27.692915 26341 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:20:27.694897 32037 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.697088 32502 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.672375 32495 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.701800 32489 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.702301 26342 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:27.705938 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.713092 26338 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:27.674249 32570 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.726063 25632 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:27.726825 26337 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:20:27.728981 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:27.729771 25635 raft_consensus.cc:2368] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:20:27.730868 32416 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.731384 32461 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.717063 32575 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.732491 32472 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.715108 32496 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:20:27.734133 25601 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f80a8201dcaf4f2a9290c47595f9cf7b. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.736508 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54831.9us
I20260709 12:20:27.737176 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:27.736433 32496 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:27.747987 32496 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.749095 32496 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:27.731402 32570 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:27.757782 25633 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
W20260709 12:20:27.734651 32462 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
W20260709 12:20:27.770380 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 150695us
I20260709 12:20:27.785488 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:27.786319 26342 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:27.787200 26344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:27.787861 26344 raft_consensus.cc:606] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
W20260709 12:20:27.789212 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 52083.9us
I20260709 12:20:27.789615 32575 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.792333 32551 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:27.794626 32559 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:27.795578 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 407586us
I20260709 12:20:27.798189 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.781448 32495 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.798936 25708 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:27.802690 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.803321 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.803922 26339 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:27.806780 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.808221 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.809520 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.812541 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:27.814224 25782 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:27.816609 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:27.817325 25785 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:27.818773 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.822948 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.823622 25783 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:27.826920 25780 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:20:27.828944 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.828727 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:27.836035 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.836629 25709 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:27.830466 32568 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:20:27.810981 32559 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.837708 32443 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:27.838416 32443 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.838801 32443 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:27.827174 32577 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:27.844647 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:27.845288 26340 raft_consensus.cc:606] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:27.845642 26340 raft_consensus.cc:614] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:27.841679 32570 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:27.801792 32095 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:20:27.847793 32443 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:27.850054 32443 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:27.858453 25779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1334) took 1278 ms. Trace:
I20260709 12:20:27.858868 25779 rpcz_store.cc:276] 0709 12:20:26.580026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.580413 (+   387us) service_pool.cc:224] Handling call
0709 12:20:26.580699 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:26.695051 (+114352us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:26.840049 (+144998us) log.cc:844] Serialized 14 byte log entry
0709 12:20:27.442941 (+602892us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180033b4a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1
0709 12:20:27.443681 (+   740us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b180033b4a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:20:27.773743 (+330062us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b180033b4a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b55374bc 00007f29b288548b 00007f29bb6df9a9 00007f29b28a169b 00007f29b289f93f 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287
0709 12:20:27.773882 (+   139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:27.773980 (+    98us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:27.774042 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:27.858203 (+ 84161us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54001a77d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:27.858252 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:27.858387 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2514,"prepare.queue_time_us":300679,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":8678,"raft.queue_time_us":56524,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":24222,"spinlock_wait_cycles":341441664,"thread_start_us":367211,"threads_started":3,"wal-append.queue_time_us":320781}
I20260709 12:20:27.861980 32570 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:27.862900 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:27.863517 25786 raft_consensus.cc:3060] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:27.865346 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.866302 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.866545 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.868570 25780 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:27.862248 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
W20260709 12:20:27.923641 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61476.5us
I20260709 12:20:27.923795 25750 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.925606 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:27.926730 25786 raft_consensus.cc:2468] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 22.
I20260709 12:20:27.926839 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.928860 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:27.929780 26340 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:27.935372 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.936125 25786 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
W20260709 12:20:27.937377 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 85592.1us
I20260709 12:20:27.938257 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.940754 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.938685 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:27.942857 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.943332 25750 leader_election.cc:400] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17.
I20260709 12:20:27.943670 25750 leader_election.cc:403] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:20:27.943768 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:27.944453 25786 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:20:27.946343 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:27.946403 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:27.951220 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:27.952991 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 377509us
I20260709 12:20:27.955559 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:27.960772 25750 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:27.959458 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:27.975754 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:27.936798 32495 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:27.942065 32399 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:27.984014 26340 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 14.
I20260709 12:20:27.987180 26339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:27.967136 25677 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:27.984817 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:27.988123 26339 raft_consensus.cc:606] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:27.951162 32445 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:27.951740 32540 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:27.986135 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:27.989140 32556 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:20:27.994138 32557 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:20:27.981710 32423 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:27.995177 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:27.979972 32250 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:20:28.002642 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45171.9us
I20260709 12:20:28.047662 25676 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:27.952343 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:28.007690 32252 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:27.981839 32549 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:20:28.014034 25785 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 17.
I20260709 12:20:27.959609 32497 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:27.962059 32472 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 21. Current term is 22.
I20260709 12:20:27.944434 32462 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:20:27.983603 32495 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:28.012988 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45970.3us
W20260709 12:20:28.045601 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50491.8us
I20260709 12:20:28.016036 32528 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:28.018595 32538 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:28.079623 32528 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:28.022514 32545 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:28.025660 32557 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:28.021595 32444 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:28.025084 32423 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.085829 32423 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:28.087337 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 1157) took 1503 ms. Trace:
I20260709 12:20:28.087862 26343 rpcz_store.cc:276] 0709 12:20:26.584189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.584321 (+   132us) service_pool.cc:224] Handling call
0709 12:20:26.589772 (+  5451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:26.590013 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:26.731967 (+141954us) log.cc:844] Serialized 14 byte log entry
0709 12:20:27.011720 (+279753us) log.cc:844] Serialized 14 byte log entry
0709 12:20:27.211767 (+200047us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:27.403597 (+191830us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b18003bea20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b5987060 00007f29b5999716 00007f29b59aa287 00007f29b59aa208
0709 12:20:27.456776 (+ 53179us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b18003bea20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:20:27.595838 (+139062us) log.cc:844] Serialized 14 byte log entry
0709 12:20:27.596174 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 12:20:27.665538 (+ 69364us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:27.665608 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:27.665672 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:27.665749 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:27.752590 (+ 86841us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b18003bea20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:20:28.087107 (+334517us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b5400582ad0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598e8ad 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:28.087260 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42584,"prepare.queue_time_us":274734,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":157,"raft.queue_time_us":472699,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":14797,"spinlock_wait_cycles":64273408,"thread_start_us":599029,"threads_started":4,"wal-append.queue_time_us":406025}
I20260709 12:20:28.059276 32547 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 16. Current term is 17.
W20260709 12:20:28.067955 32582 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:20:28.068387 32542 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:28.017048 32399 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.057832 32252 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:28.103232 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:28.070674 32503 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:28.071328 32579 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:28.074640 32495 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:28.055131 32445 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:28.078979 32549 consensus_queue.cc:1243] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer 0ae979356d694a09be4b0cd976ea354a log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:20:28.081099 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 88823.5us
I20260709 12:20:28.083379 32444 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.066058 32497 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.083454 32528 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.080394 32545 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:28.081262 32584 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:20:28.074318 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:28.095371 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:28.109651 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 63551.6us
I20260709 12:20:28.141086 25709 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:28.141798 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:28.099673 32507 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:20:28.102391 32542 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:28.033342 32530 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.092541 32557 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:28.098767 32569 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:28.133929 25677 leader_election.cc:400] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
W20260709 12:20:28.113790 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51994.6us
W20260709 12:20:28.144835 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 49598.8us
I20260709 12:20:28.110373 32503 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:28.157279 32503 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:28.157321 25676 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:28.159991 26307 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:28.160873 32259 raft_consensus.cc:993] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:28.116238 32445 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.161422 32445 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:28.161809 32504 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:28.163622 32473 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:28.166441 26307 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:28.167047 32504 raft_consensus.cc:1081] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:28.168543 25677 leader_election.cc:403] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:20:28.152809 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 192149us
I20260709 12:20:28.170549 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:28.171446 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54513.5us
I20260709 12:20:28.173024 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:28.171793 25750 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:28.169544 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:28.174662 32250 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:28.175285 32250 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:28.176118 32484 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:28.170701 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:28.142925 32545 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:28.176388 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:28.179206 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:28.150339 32542 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:28.147812 32560 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:28.152719 32551 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:28.120270 32394 consensus_queue.cc:1059] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:20:28.207525 25784 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:28.155522 32422 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.177166 32538 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:28.177573 32294 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:28.180958 32377 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:20:28.179689 32504 raft_consensus.cc:2804] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:28.199920 32541 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:20:28.211361 32584 log.cc:927] Time spent T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:20:28.203017 32556 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:28.282958 32541 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.283457 32556 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.283681 32541 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:28.284053 32556 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:28.287216 32399 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:28.288164 32504 raft_consensus.cc:697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:28.289163 32504 consensus_queue.cc:237] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.195676 32555 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:28.207819 32259 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:20:28.265815 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 95202.3us
I20260709 12:20:28.216002 32529 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:28.207788 32550 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:28.210018 32037 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:28.215457 32551 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:28.267081 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:28.304447 25676 leader_election.cc:400] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 14. Current term is 15.
I20260709 12:20:28.179404 32569 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.176350 32250 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.169559 32445 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.326155 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:28.273224 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51390.4us
W20260709 12:20:28.327214 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 185494us
I20260709 12:20:28.340206 25676 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:28.218617 32560 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.346356 25782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:28.347215 25782 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:28.347584 25782 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:28.348505 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:28.349238 26339 raft_consensus.cc:2368] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:20:28.350153 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:28.351079 25713 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:28.352427 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:28.261291 32422 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.350979 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:28.354349 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:28.303177 32259 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:28.356046 25709 raft_consensus.cc:2393] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 18: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:28.328277 32551 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:28.355896 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:28.323650 32558 raft_consensus.cc:3055] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:28.370396 32558 raft_consensus.cc:740] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:28.371480 32558 consensus_queue.cc:260] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.373023 32558 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:20:28.338588 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 141693us
I20260709 12:20:28.375739 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet ef4fd24cfa184aa8a8a1de8f32296e6a (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:28.316627 32294 consensus_queue.cc:1048] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:20:28.386059 32294 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:28.389501 32294 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:28.257938 32252 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.367319 25782 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.355343 32550 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:28.399947 25706 raft_consensus.cc:2393] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 12: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:28.322152 32529 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.368263 32259 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:28.324848 32503 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:28.450131 25782 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:28.421932 32586 log.cc:927] Time spent T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:20:28.405247 32550 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:28.484465 32556 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.486021 25707 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 12:20:28.462211 32599 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:20:28.466656 32550 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:28.492063 32422 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:28.506968 32252 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:28.454176 32587 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
W20260709 12:20:28.519795 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 167035us
I20260709 12:20:28.422886 32497 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:28.459003 32535 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:20:28.521409 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:28.526821 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:28.527491 25707 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:28.502112 32556 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:28.528764 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:28.529798 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 309702us
I20260709 12:20:28.530184 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:28.530795 25709 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:28.530905 25706 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:28.532466 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:28.533978 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:28.535759 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.455492 32399 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.540652 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:28.559700 25707 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:28.525611 32473 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:28.529816 32444 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:28.541783 32464 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:28.542090 32530 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:28.550104 32489 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:20:28.549880 32399 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:28.620764 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 79675.9us
I20260709 12:20:28.645804 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19825207f8744cf7809ab169c0ec8973" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:28.646507 26336 raft_consensus.cc:3055] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:28.646853 26336 raft_consensus.cc:740] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:28.646741 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:28.650615 32399 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:28.651322 25709 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:28.652623 32529 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:28.653378 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e959328c44aed94ce17546b6198e2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:28.656720 32583 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.613s	user 0.004s	sys 0.000s
I20260709 12:20:28.626155 32495 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:28.663939 32589 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
I20260709 12:20:28.664695 32495 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:28.605105 32423 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.668447 32423 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:28.577668 32473 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.670440 32473 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:28.672322 32610 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:28.674064 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:28.674736 25707 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:28.674947 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:28.675195 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:28.647804 26336 consensus_queue.cc:260] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.678882 32445 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:28.679818 32598 log.cc:927] Time spent T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 12:20:28.681278 26337 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:28.681691 32503 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:28.674268 25752 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:28.679823 32530 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:20:28.686239 32577 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:28.686926 32577 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.635915 32439 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:28.688540 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:28.689661 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:28.689668 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:28.691627 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
W20260709 12:20:28.696143 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 360269us
I20260709 12:20:28.697681 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:28.699644 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 432677us
I20260709 12:20:28.701043 25603 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:28.703773 32507 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:28.682909 32612 raft_consensus.cc:2804] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:28.704136 32541 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.704548 32612 raft_consensus.cc:697] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:28.708917 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:28.713315 25707 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:28.721064 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:28.726979 25635 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 12:20:28.730603 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:28.733660 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:28.716926 25752 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:28.743594 32250 raft_consensus.cc:2804] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:28.700140 32095 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:28.695890 32037 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:28.755579 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58795.6us
W20260709 12:20:28.757652 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 56686.7us
I20260709 12:20:28.759790 32612 consensus_queue.cc:237] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.763144 26337 raft_consensus.cc:1240] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 12:20:28.764118 32579 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:28.697304 32484 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:20:28.797291 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:28.793709 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:28.797379 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:28.794714 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:28.830466 25706 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:28.830772 26337 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:20:28.852430 25752 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:28.853732 32530 raft_consensus.cc:2804] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:28.840261 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:28.855448 25711 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 20: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:28.858688 32530 raft_consensus.cc:697] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:28.858340 25780 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:28.860150 32530 consensus_queue.cc:237] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.793215 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:28.750353 32095 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:28.762034 32601 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:28.786023 32250 raft_consensus.cc:697] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:28.746166 32444 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
W20260709 12:20:28.916308 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 161945us
I20260709 12:20:28.871297 32095 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:28.916568 32551 consensus_meta.cc:287] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.006s
I20260709 12:20:28.913115 32633 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:28.964040 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 215822us
I20260709 12:20:28.969755 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:28.934368 32569 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:28.985569 32550 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:28.992370 32542 consensus_meta.cc:287] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.000s
I20260709 12:20:28.992937 32542 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.997668 25786 raft_consensus.cc:1240] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:29.003991 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:28.975797 32616 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:20:28.976698 32608 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:28.977581 32544 raft_consensus.cc:993] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:29.008015 32550 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.004444 32250 consensus_queue.cc:237] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.041973 32544 raft_consensus.cc:1081] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
W20260709 12:20:29.028559 32591 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:20:28.969552 32633 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:28.936921 32037 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.043460 32445 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:29.043992 32445 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.045253 32633 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:29.040233 32545 consensus_meta.cc:287] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.831s	user 0.007s	sys 0.005s
I20260709 12:20:29.010097 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:29.012562 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:29.015930 25706 raft_consensus.cc:1240] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:29.016944 25707 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:20:29.051991 32545 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.055054 32545 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.046175 32445 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:29.067718 32443 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:29.068346 32443 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.072793 32600 consensus_queue.cc:1059] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:29.073884 32465 raft_consensus.cc:3055] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:29.074433 32465 raft_consensus.cc:740] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:29.075901 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 66064.2us
I20260709 12:20:29.075626 32465 consensus_queue.cc:260] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.031401 32608 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.008489 32577 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.007823 32472 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:29.007257 32557 consensus_meta.cc:287] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.826s	user 0.010s	sys 0.002s
I20260709 12:20:29.037253 32095 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.079728 32557 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:28.949640 32551 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.081802 32095 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:28.995802 32542 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:29.009076 32507 consensus_queue.cc:1059] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:20:29.083302 32507 raft_consensus.cc:3055] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:29.083438 32551 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:29.083768 32507 raft_consensus.cc:740] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:29.084295 25785 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:29.085285 32560 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:29.087141 32259 consensus_meta.cc:287] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 12:20:29.087574 32259 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.088388 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.090909 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:29.091899 25713 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 21: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:29.079984 32608 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.093235 32443 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.093860 32443 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:29.094863 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:29.077803 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.082266 32557 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.098802 32465 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:29.112787 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:29.113154 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:29.113399 25637 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:29.114228 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:29.114905 25633 raft_consensus.cc:2393] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 12: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:29.115249 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:29.085597 32416 consensus_queue.cc:1059] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:20:29.121012 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.118341 25785 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:29.118341 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:29.123507 25630 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:29.125936 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:29.084743 32507 consensus_queue.cc:260] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.117777 32484 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:29.132433 32484 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:29.133342 32484 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.134779 32484 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:29.127892 32259 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.136861 25631 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:29.142570 32570 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:29.143195 32570 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.095398 32541 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.102778 25676 leader_election.cc:400] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 13. Current term is 14.
I20260709 12:20:29.151351 32647 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:29.160626 32647 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.163175 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.099689 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.103696 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:29.105083 32439 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.105798 26343 raft_consensus.cc:3060] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:29.109684 25712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:52030
I20260709 12:20:29.167317 25707 raft_consensus.cc:2368] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:29.167418 25712 raft_consensus.cc:606] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:29.167987 25712 raft_consensus.cc:614] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:29.159463 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50966 (request call id 2565) took 1605 ms. Trace:
I20260709 12:20:29.168569 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.104532 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.110357 25704 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:20:29.169550 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.112110 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:29.170701 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:29.095479 25787 raft_consensus.cc:3060] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:29.171147 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:29.168926 25784 rpcz_store.cc:276] 0709 12:20:27.554366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.554506 (+   140us) service_pool.cc:224] Handling call
0709 12:20:29.159181 (+1604675us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400086bd4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:29.159349 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1399248128}
I20260709 12:20:29.152299 25708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3549) took 1203 ms. Trace:
I20260709 12:20:29.172381 25630 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:20:29.171952 25708 rpcz_store.cc:276] 0709 12:20:27.948669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.948954 (+   285us) service_pool.cc:224] Handling call
0709 12:20:27.949227 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:28.121887 (+172660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:28.121938 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:28.122003 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:28.122053 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:29.152067 (+1030014us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54000744d4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:29.152117 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:29.152239 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396467456}
I20260709 12:20:29.158471 26336 raft_consensus.cc:3060] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:29.158200 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55625.7us
I20260709 12:20:29.182564 25676 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:29.201020 25633 raft_consensus.cc:2368] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:29.201951 25637 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:29.203423 25677 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:29.210649 32647 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:29.216624 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.219733 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:29.220432 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:29.224339 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.225045 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:29.226518 25635 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:20:29.226799 26344 raft_consensus.cc:3060] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:29.226924 25635 pending_rounds.cc:85] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Aborting all ops after (but not including) 13
I20260709 12:20:29.227422 25635 pending_rounds.cc:107] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:20:29.227707 25635 raft_consensus.cc:2889] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:29.227715 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.230176 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:29.230829 26337 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:29.230816 26336 raft_consensus.cc:2468] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
W20260709 12:20:29.231825 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49412.3us
I20260709 12:20:29.232615 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:29.239100 25752 leader_election.cc:400] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:29.239614 25752 leader_election.cc:403] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:20:29.241119 32549 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:29.241807 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:29.242233 32549 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.242470 32555 raft_consensus.cc:2804] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:29.243227 32555 raft_consensus.cc:697] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:29.244202 32555 consensus_queue.cc:237] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.246863 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:29.203002 32259 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:29.260161 32259 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.274480 32638 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.274955 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.275280 32638 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:29.276652 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52377.4us
I20260709 12:20:29.155505 32579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:41738 (request call id 1) took 2906 ms. Trace:
I20260709 12:20:29.277544 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.285894 26335 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
W20260709 12:20:29.287056 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45334.6us
I20260709 12:20:29.287900 25787 raft_consensus.cc:2468] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:29.287933 26344 raft_consensus.cc:2468] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
W20260709 12:20:29.286608 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 66610.5us
I20260709 12:20:29.290026 25601 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.290894 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9eec1cb847cc4587be1133c287c515b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.291131 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.287448 26343 raft_consensus.cc:2468] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:29.293639 25676 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:29.299944 26341 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
I20260709 12:20:29.302809 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.306908 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:29.307849 25706 raft_consensus.cc:2368] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13.
I20260709 12:20:29.288705 32612 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:29.321810 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:29.323195 26307 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:29.323948 26337 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:29.325103 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:29.290252 32530 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:29.329360 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.330451 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.304880 32394 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:29.306733 32578 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:29.333635 25787 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:29.324395 32423 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:20:29.313992 32399 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.336221 25784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:29.319036 32575 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.334509 25780 raft_consensus.cc:1240] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:29.332870 26339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:29.338295 25676 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:29.317888 32250 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:29.336335 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:29.322433 32377 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:29.327311 32559 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.349761 26339 raft_consensus.cc:606] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:29.376634 32538 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:29.377094 32639 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:29.377204 32538 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.377698 32639 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:29.378659 32639 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.379554 32538 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.332412 32632 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:29.380398 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.374316 26339 raft_consensus.cc:614] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:29.346256 25784 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:29.384048 25784 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:29.397524 32547 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:29.398427 32547 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.347810 32294 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:29.345866 32633 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:29.344758 32636 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:29.347275 32578 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.429665 32578 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.370548 32612 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:20:29.437749 32636 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.456923 32636 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:29.345158 32637 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:20:29.383148 32377 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.371330 32575 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.466369 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:29.466419 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:29.480015 25787 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:29.414227 32399 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:29.474148 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:29.496125 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 157909us
I20260709 12:20:29.496328 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:29.497337 25635 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:29.439254 32580 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.498617 25784 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.501724 26307 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:29.500398 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 148254us
I20260709 12:20:29.507495 32655 ts_tablet_manager.cc:933] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:20:29.509027 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:29.509402 25784 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.511258 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:29.515097 32648 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:20:29.512979 32496 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.285449 32579 rpcz_store.cc:276] 0709 12:20:26.248652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.248862 (+   210us) service_pool.cc:224] Handling call
0709 12:20:26.564731 (+315869us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b4c004b6300. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b3c5ba20 00007f29b3c6ca29 00007f29b3c32644 00007f29b3c59f2d 00007f29b3c59b93 00007f29b58e81af 00007f29b58e7f3e 00007f29b596d888 00007f29b598a75c 00007f29b598b76a 00007f29b598a91b
0709 12:20:26.565411 (+   680us) log.cc:844] Serialized 506 byte log entry
0709 12:20:26.688296 (+122885us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b54000c9520. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b590fb26 00007f29b590949a 00007f29b5908714 00007f29b58e8eaa 00007f29b58efd53 00007f29b58efcd9 00007f29b58efc69 00007f29b58efc31 00007f29b58eedfd 0000000000388634
0709 12:20:26.776441 (+ 88145us) spinlock_profiling.cc:243] Waited 6.99 ms on lock 0x7b1800324520. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b590fb26 00007f29b590949a 00007f29b5908714
0709 12:20:26.776932 (+   491us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b54000c9520. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b590fb26 00007f29b590949a 00007f29b5908714 00007f29b58e8eaa 00007f29b58efd53 00007f29b58efcd9 00007f29b58efc69 00007f29b58efc31 00007f29b58eedfd 0000000000388634
0709 12:20:27.404349 (+627417us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b54000c9520. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b5908142 00007f29b58e8eaa 00007f29b58efd53 00007f29b58efcd9 00007f29b58efc69 00007f29b58efc31 00007f29b58eedfd 0000000000388634 0000000000388479 00007f29b28a2475
0709 12:20:29.155404 (+1751055us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":264,"mutex_wait_us":4982,"raft.queue_time_us":8851,"raft.run_cpu_time_us":7957,"raft.run_wall_time_us":955114,"spinlock_wait_cycles":97913856,"thread_start_us":66379,"threads_started":1,"wal-append.queue_time_us":146001,"wal-append.run_cpu_time_us":2989,"wal-append.run_wall_time_us":1225083}
I20260709 12:20:29.518173 32570 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.525902 32377 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.531183 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:29.532320 32567 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.510588 32547 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.533772 25601 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:29.535079 32653 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:20:29.535717 32567 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:29.537037 32575 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.540159 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.503566 32580 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.548527 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:29.549379 32580 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.549419 25634 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:29.552475 25751 leader_election.cc:400] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:29.553056 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.553587 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:29.555156 32637 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:29.555275 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.522452 32496 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:29.557224 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.565840 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.567447 32567 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:29.567962 32567 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:29.543469 32638 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.570300 32637 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:29.577561 25634 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:29.561304 32636 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:20:29.586818 25634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:29.588958 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:29.592777 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.568847 32567 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:29.594938 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:29.598730 25751 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:29.601416 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:29.616688 32444 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.616492 32529 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:29.597281 32560 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.622438 32529 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:29.629184 32620 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.706s	user 0.003s	sys 0.000s
I20260709 12:20:29.632187 25778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50194 (request call id 1354) took 1286 ms. Trace:
I20260709 12:20:29.632591 25778 rpcz_store.cc:276] 0709 12:20:28.345915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.351168 (+  5253us) service_pool.cc:224] Handling call
0709 12:20:28.351460 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:28.549979 (+198519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:28.550016 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:28.564695 (+ 14679us) log.cc:844] Serialized 26 byte log entry
0709 12:20:28.620017 (+ 55322us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:28.620056 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:29.163846 (+543790us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b5400320854. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:29.163896 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:29.628806 (+464910us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b2400243cc8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27289c4 00007f29b2726acb 00007f29b28e1dc9 00007f29b28e1a55 00007f29b28e109d 00007f29b551310e 00007f29b552a88c 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221
0709 12:20:29.631884 (+  3078us) raft_consensus.cc:1685] finished
0709 12:20:29.631920 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:29.632119 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68426,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":125,"spinlock_wait_cycles":139827840,"thread_start_us":198927,"threads_started":2,"wal-append.queue_time_us":192021}
I20260709 12:20:29.634914 32567 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:29.635488 25601 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.637328 32660 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:29.630376 32529 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.637884 32660 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:29.638885 32660 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.679020 32663 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:29.679708 32259 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:29.679808 32663 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.671214 32655 tablet_copy_client.cc:323] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:20:29.687844 32663 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.706667 25634 raft_consensus.cc:606] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:29.707208 25634 raft_consensus.cc:614] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:29.725747 25679 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.733282 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.733508 25707 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:29.735301 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742
I20260709 12:20:29.749357 25709 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:20:29.754734 32605 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:29.755309 32605 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:29.755698 32605 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:29.764523 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.771016 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.771759 25710 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 15.
I20260709 12:20:29.777504 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:20:29.781879 25710 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:29.764557 32444 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:20:29.794072 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.787461 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:29.795360 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 54130.9us
I20260709 12:20:29.790416 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:29.798755 25712 raft_consensus.cc:1240] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:29.798975 25752 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.803349 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:29.805204 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:29.806051 25709 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:29.802860 32560 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:29.799885 32547 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:20:29.801241 32542 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 12:20:29.825842 32650 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.386s	user 0.003s	sys 0.001s
I20260709 12:20:29.828637 25783 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:29.814446 25603 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:29.835393 32416 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:29.836886 32507 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:29.847508 32605 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.849685 26344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1550) took 1203 ms. Trace:
W20260709 12:20:29.850045 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 46736.3us
I20260709 12:20:29.851222 26344 rpcz_store.cc:276] 0709 12:20:28.646698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.646822 (+   124us) service_pool.cc:224] Handling call
0709 12:20:29.849507 (+1202685us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b5400573d54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:29.849630 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317099904}
I20260709 12:20:29.861469 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:29.862641 32612 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:29.867707 32605 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:29.869069 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet f80a8201dcaf4f2a9290c47595f9cf7b from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:20:29.869648 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet f80a8201dcaf4f2a9290c47595f9cf7b from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-f80a8201dcaf4f2a9290c47595f9cf7b
I20260709 12:20:29.872200 32648 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:29.872892 32648 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:29.873384 32648 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:29.877750 25601 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.886076 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:29.921634 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 419902us
I20260709 12:20:29.926460 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:29.927038 25785 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:29.931931 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 54272.9us
W20260709 12:20:29.931979 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 47088.2us
I20260709 12:20:29.933389 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:29.940227 32627 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.942164 25635 raft_consensus.cc:1240] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:29.942586 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:29.943151 26344 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 12.
I20260709 12:20:29.947077 26341 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:29.969492 32416 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:29.977141 32144 consensus_queue.cc:1059] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:29.978575 32144 raft_consensus.cc:3055] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:29.978925 32144 raft_consensus.cc:740] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:29.979663 32144 consensus_queue.cc:260] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:29.980739 32144 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:29.983667 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:29.986049 32416 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:29.986564 32416 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.986972 32416 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:29.796911 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:29.942070 32508 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:29.989837 32508 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
W20260709 12:20:29.990968 26343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56168 (request call id 1553) took 1270 ms (client timeout 1577 ms). Trace:
W20260709 12:20:29.991367 26343 rpcz_store.cc:269] 0709 12:20:28.720563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.720734 (+   171us) service_pool.cc:224] Handling call
0709 12:20:29.990800 (+1270066us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54005a0354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b59932fa 00007f29ba954b17 00007f29b3ecf6d4 00007f29b3ecf662 00007f29b3ecf591 00007f29b3ecf50c 00007f29b3ece7c2 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:29.990910 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882983680}
I20260709 12:20:29.991253 32508 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.937940 32473 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:29.946501 32632 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:30.004956 32632 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.007495 32632 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:29.954342 32575 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.009011 32575 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:29.940781 32627 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.033543 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:30.033798 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:30.034271 26337 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:30.034595 26337 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:30.035362 26337 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.041524 26337 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:30.057816 26336 raft_consensus.cc:1275] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:30.063298 26344 raft_consensus.cc:1275] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:20:30.061815 25633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.065s	sys 0.147s
W20260709 12:20:30.064195 25633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.066s	sys 0.147s
I20260709 12:20:30.086841 25750 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:30.046075 32422 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.099203 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:30.109011 26344 raft_consensus.cc:1275] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:30.057534 32641 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:30.111517 32529 consensus_queue.cc:1048] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:20:30.123415 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:30.073192 32660 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:30.105063 32621 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.117620 32472 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.144447 32530 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:30.173915 25634 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:30.112825 32422 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.118060 32559 consensus_queue.cc:1048] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:20:30.186700 32549 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:30.190017 32422 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:30.194581 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:30.157051 32575 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:30.130412 32641 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:30.211333 32627 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:30.152240 32647 consensus_queue.cc:1048] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:20:30.229622 26337 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:30.245649 26336 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:30.166121 32472 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.258355 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:30.260933 26309 leader_election.cc:400] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13.
I20260709 12:20:30.261478 26309 leader_election.cc:403] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:20:30.276348 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:30.270846 32575 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:30.270227 32462 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 12. Current term is 13.
I20260709 12:20:30.273355 32416 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.273515 32648 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.293706 26338 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:30.271262 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:30.295563 32663 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:20:30.296221 32663 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:30.297140 32663 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.303285 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:30.306864 25710 raft_consensus.cc:1275] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:30.309541 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:30.312498 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:30.316679 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:30.331590 32580 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:20:30.369210 25786 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:30.370597 32294 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.370755 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:30.371172 32294 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:30.373370 32294 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:30.380774 32507 consensus_meta.cc:287] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
I20260709 12:20:30.384173 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:30.359615 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 62176.8us
I20260709 12:20:30.387318 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.389297 25710 raft_consensus.cc:2410] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:20:30.388447 32627 consensus_queue.cc:1059] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:20:30.406690 32552 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:30.419257 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 357386us
I20260709 12:20:30.410562 32416 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:30.440744 32648 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:30.434358 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:30.443661 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:30.419819 32472 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:30.452195 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.462711 25779 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:30.451121 25648 tablet_copy_source_session.cc:215] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:30.561866 32685 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:30.574595 32685 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.576977 32685 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:30.564046 32578 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:30.581540 32578 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:30.584050 32144 consensus_meta.cc:287] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.006s
I20260709 12:20:30.611739 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:30.615711 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 228944us
W20260709 12:20:30.633692 25584 auto_rebalancer.cc:629] Failed to schedule move for tablet 0024aee53c2a47e59a06e7e69bfb9eb2: Illegal state: Replica c7bf8f20701245bcbaa09cc663025f97 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:30.640002 32693 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:30.640887 32693 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.643077 32693 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:30.644299 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.648509 32521 raft_consensus.cc:3055] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:30.660953 32521 raft_consensus.cc:740] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:30.662024 32578 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:30.661852 32521 consensus_queue.cc:260] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.662889 32521 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:30.663866 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:30.664294 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:30.665155 26343 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:30.672652 26342 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:20:30.683491 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:30.686682 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:30.687826 26343 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:30.688544 32696 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:30.689141 32696 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.693405 32478 consensus_queue.cc:1048] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:30.695729 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:30.682672 32580 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:30.696556 32478 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:30.716147 32696 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:30.728439 32641 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:30.718518 32545 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.714087 32508 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:20:30.731386 32545 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:30.732350 32660 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:30.732898 32660 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:30.735981 32545 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:30.745080 32660 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:30.751082 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.751924 25634 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:30.752868 32660 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:30.753222 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:30.754254 32552 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:30.754832 32552 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.755309 32552 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:30.759882 32592 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:30.760522 32592 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:30.761023 32551 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:30.761667 32551 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:30.763768 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:30.764751 32551 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:30.765448 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:30.708992 32580 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:30.771126 32580 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:30.777937 25636 raft_consensus.cc:1240] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:30.778023 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.784406 25786 raft_consensus.cc:1275] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:30.790134 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:30.790894 25780 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:30.792831 32655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f80a8201dcaf4f2a9290c47595f9cf7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.797796 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:30.797883 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:30.807268 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 108419us
I20260709 12:20:30.822114 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:30.827986 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:30.832505 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:30.834882 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.842924 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:30.843650 25782 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:30.846181 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.858003 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.858296 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.860800 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.872103 25631 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:30.873600 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:30.826882 32592 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:30.877396 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:30.840893 32580 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:30.882429 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:30.905192 25636 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:30.906737 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:30.907579 32632 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:30.883570 32405 consensus_queue.cc:1059] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:20:30.910953 32405 raft_consensus.cc:3055] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:30.911423 32405 raft_consensus.cc:740] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:30.912472 32405 consensus_queue.cc:260] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.914003 32405 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:30.878334 32648 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:30.936510 25712 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 12:20:30.937107 25712 pending_rounds.cc:85] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 12
I20260709 12:20:30.937368 25712 pending_rounds.cc:107] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 12:20:30.937681 25712 raft_consensus.cc:2889] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:30.872432 32639 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:30.953377 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 71239us
I20260709 12:20:30.940949 32580 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:30.955730 32632 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:30.882417 32551 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 12:20:30.930238 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56146 (request call id 1184) took 1709 ms. Trace:
W20260709 12:20:30.988505 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 276650us
I20260709 12:20:31.009630 32580 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.966634 32639 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:30.951777 32552 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:20:31.046360 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 50548.2us
I20260709 12:20:31.016644 32575 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:31.057960 32670 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.806s	user 0.001s	sys 0.004s
I20260709 12:20:31.049887 32464 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.063675 32570 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:31.064637 32632 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.067159 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:31.069837 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.086306 25712 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:31.115531 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.116163 25704 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
W20260709 12:20:31.124311 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.128691 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:31.131645 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 289588us
I20260709 12:20:31.142390 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:31.111550 32521 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:31.143919 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 252847us
I20260709 12:20:31.146178 25603 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:31.149312 25633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51520 (request call id 2883) took 1599 ms. Trace:
I20260709 12:20:31.159087 25633 rpcz_store.cc:276] 0709 12:20:29.549742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.549986 (+   244us) service_pool.cc:224] Handling call
0709 12:20:29.550317 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:29.558369 (+  8052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:29.558401 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:29.558937 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:20:30.064719 (+505782us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:20:30.064775 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:30.064881 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:31.057119 (+992238us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:31.145110 (+ 87991us) raft_consensus.cc:1685] finished
0709 12:20:31.145134 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:31.149227 (+  4093us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7649,"raft.queue_time_us":8466,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":3489408,"thread_start_us":505336,"threads_started":1,"wal-append.queue_time_us":606340}
I20260709 12:20:31.112003 32464 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:31.229225 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 74186.5us
W20260709 12:20:31.227958 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 244114us
I20260709 12:20:31.159339 32567 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:31.160074 32462 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:31.248699 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.313097 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.262883 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:31.337644 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 69962.1us
W20260709 12:20:31.337148 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75570.1us
I20260709 12:20:30.989715 32655 rpcz_store.cc:276] 0709 12:20:29.220891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.221394 (+   503us) service_pool.cc:224] Handling call
0709 12:20:29.507885 (+286491us) ts_tablet_manager.cc:934] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
0709 12:20:30.785761 (+1277876us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:20:30.909883 (+124122us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:30.910041 (+   158us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:20:30.929171 (+ 19130us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:20:30.929598 (+   427us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:30.930038 (+   440us) ts_tablet_manager.cc:989] Replica registered.
0709 12:20:30.930126 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":21917,"tablet-copy.queue_time_us":189988,"thread_start_us":296509,"threads_started":2}
I20260709 12:20:31.375401 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.381211 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:31.234874 32592 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.159451 32521 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:31.391388 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 253985us
I20260709 12:20:31.392429 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:31.234405 32268 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.141s
I20260709 12:20:31.312467 32567 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:31.424175 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:31.441757 25782 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:31.380420 32633 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.391875 32709 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.423030 32567 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:31.474017 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.475355 25780 raft_consensus.cc:2393] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 18: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:31.422258 32715 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.398028 32592 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.473173 32709 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.479946 32521 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.473423 32655 tablet_copy_client.cc:806] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:31.481698 32464 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:31.483340 32680 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.159s	user 0.007s	sys 0.013s
I20260709 12:20:31.468585 32633 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.450450 32439 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.452610 32552 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.465178 32507 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:31.499050 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 117970us
I20260709 12:20:31.532199 25705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 3204) took 1654 ms. Trace:
I20260709 12:20:31.511749 32655 tablet_copy_client.cc:670] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:31.515373 32439 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.532676 25705 rpcz_store.cc:276] 0709 12:20:29.877817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.878006 (+   189us) service_pool.cc:224] Handling call
0709 12:20:29.878552 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:29.879011 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:29.879054 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:29.879543 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:20:30.058754 (+179211us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:30.058797 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:30.058870 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:31.430055 (+1371185us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:31.531863 (+101808us) raft_consensus.cc:1685] finished
0709 12:20:31.531889 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:31.532124 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":177,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":54135,"spinlock_wait_cycles":4930432,"thread_start_us":137587,"threads_started":1,"wal-append.queue_time_us":357764}
I20260709 12:20:31.535266 32439 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:31.554451 25787 raft_consensus.cc:1240] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:31.489112 32715 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:31.573369 32710 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:20:31.576061 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.577414 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.579974 25706 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:20:31.580276 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 241820us
I20260709 12:20:31.538306 32723 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.553033 32552 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.587435 32717 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:31.588587 25750 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:31.526211 32507 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.591325 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:31.595336 25631 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:31.603632 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.578948 32579 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:31.566356 32730 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:20:31.592363 32708 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 12:20:31.615545 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:31.593757 32704 log.cc:927] Time spent T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 12:20:31.594218 32552 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:31.635742 32678 log.cc:927] Time spent T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 1.441s	user 0.003s	sys 0.008s
I20260709 12:20:31.636874 25711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 3201) took 1770 ms. Trace:
I20260709 12:20:31.637251 25711 rpcz_store.cc:276] 0709 12:20:29.866281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.866449 (+   168us) service_pool.cc:224] Handling call
0709 12:20:29.866777 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:29.867103 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:29.867135 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:29.867580 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:20:30.061028 (+193448us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:30.061075 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:30.061136 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:31.635081 (+1573945us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:31.636602 (+  1521us) raft_consensus.cc:1685] finished
0709 12:20:31.636619 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:31.636817 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2742,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":96,"spinlock_wait_cycles":14195584,"thread_start_us":193150,"threads_started":1,"wal-append.queue_time_us":289523}
I20260709 12:20:31.640100 32552 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.640833 32464 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:31.608395 32713 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:31.677160 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 284841us
I20260709 12:20:31.650549 32592 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.622727 32579 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.685778 26337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1575) took 1434 ms. Trace:
I20260709 12:20:31.686234 26337 rpcz_store.cc:276] 0709 12:20:30.251059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:30.251643 (+   584us) service_pool.cc:224] Handling call
0709 12:20:30.252880 (+  1237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:30.253286 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:30.253314 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:30.749809 (+496495us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b2c0009f618. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b4288a11 00007f29b4298372 00007f29b59caa01 00007f29b598f43a 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2
0709 12:20:30.750272 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:20:31.017880 (+267608us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:31.017939 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.018032 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:31.681474 (+663442us) raft_consensus.cc:1685] finished
0709 12:20:31.681498 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:31.685701 (+  4203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2422,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5252,"spinlock_wait_cycles":29491072,"thread_start_us":267033,"threads_started":1,"wal-append.queue_time_us":408681}
I20260709 12:20:31.590229 32723 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.653407 32733 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:31.653407 32575 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.663794 32613 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.680229 32593 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:31.692004 25676 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:31.680032 32623 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.697086 32736 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.681077 32641 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.698638 32633 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:31.667639 32713 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:31.700282 32718 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:20:31.702188 26340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56168 (request call id 1577) took 1373 ms. Trace:
I20260709 12:20:31.702086 32575 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.666285 32268 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:20:31.736286 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.750286 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.732837 25676 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:31.703259 32593 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.727841 32724 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.743276 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:31.741262 26340 rpcz_store.cc:276] 0709 12:20:30.329161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:30.329311 (+   150us) service_pool.cc:224] Handling call
0709 12:20:30.329693 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.010845 (+681152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.010881 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.011416 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:20:31.389284 (+377868us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:31.389339 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.647029 (+257690us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54005a0354. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:31.647117 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:31.701851 (+ 54734us) raft_consensus.cc:1685] finished
0709 12:20:31.701873 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:31.702122 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":576,"raft.queue_time_us":4916,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5068,"spinlock_wait_cycles":1565444992,"thread_start_us":377405,"threads_started":1,"wal-append.queue_time_us":588008}
I20260709 12:20:31.771956 32724 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.704049 32663 consensus_queue.cc:1048] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 12:20:31.773931 32703 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.533s	user 0.001s	sys 0.000s
I20260709 12:20:31.756300 32575 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.761596 32723 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.704543 32613 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.771680 32593 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:31.779198 32613 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.738319 32660 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:31.781653 32660 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.729660 32739 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:31.703249 32733 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.782428 32575 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.736287 32709 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.788921 32575 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.727465 32727 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.791878 32727 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.783933 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.794008 32727 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.784377 32660 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.768745 32713 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.771104 32726 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.754884 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:31.784768 32739 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.798467 32739 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:31.798862 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:31.799510 25784 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:31.765779 32579 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.787289 32733 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.802980 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.807217 32507 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.808802 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.811664 25631 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:31.812768 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.812827 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.812865 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.815649 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:31.816773 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.823690 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.825203 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.836211 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.836410 25784 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:31.836843 25704 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:31.848469 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.849867 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.864478 25677 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:31.866639 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.870729 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.873801 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.877401 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:31.877777 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:31.878544 25785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.879799 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:31.880141 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.879833 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.882242 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:31.878767 26340 raft_consensus.cc:2410] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:20:31.867548 25711 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:31.891841 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:31.891860 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:31.894361 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.895306 25707 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 21.
I20260709 12:20:31.896062 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.831043 32580 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.905367 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.788957 32259 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.908087 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.743239 32567 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.834765 32551 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:31.833787 32575 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.952853 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.919551 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:31.886853 32732 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.863103 32507 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.747287 32623 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:31.863284 32726 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:31.907395 32478 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:31.895989 32724 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.949728 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.916479 32472 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:31.905047 32727 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:31.954802 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:31.915576 32694 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:31.966549 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:31.740949 32736 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:31.977941 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 219108us
W20260709 12:20:31.997530 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 102056us
I20260709 12:20:31.920406 32580 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.937306 32259 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.050838 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:31.977167 32478 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:32.055171 26340 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:31.961994 32551 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:31.983371 32753 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.979990 32726 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.074852 32580 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:32.059880 32478 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:32.076341 32726 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.078166 32639 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.083308 32567 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:31.974222 32732 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.085656 32732 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:31.976280 32507 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.075917 32736 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.051651 32545 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.060272 32472 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:31.997905 32593 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.090451 32545 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.099303 32593 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.099960 32545 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:32.087755 32753 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.104084 32753 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.052157 32694 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.089669 32739 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:32.158119 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 218107us
I20260709 12:20:32.166320 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.168885 25631 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:32.169626 25705 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
W20260709 12:20:32.173483 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 250868us
W20260709 12:20:32.050419 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67110.6us
I20260709 12:20:32.182199 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:32.189007 25784 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:32.191802 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.194190 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:32.202406 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:32.217744 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:32.221417 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:32.223852 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:32.225977 32294 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:32.226622 32472 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:32.229228 32259 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.090540 32507 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.239033 32549 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.240131 32461 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:32.241060 32552 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:32.241472 32623 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:32.242542 32592 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.253733 32694 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.321126 32549 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.326870 32648 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.327749 32551 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.359853 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.360349 32545 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:32.360929 32545 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.361353 32545 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.362721 32715 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.363473 32592 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.366577 32641 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:32.363473 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 474290us
I20260709 12:20:32.366854 32592 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.370932 32648 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:32.372841 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 149349us
I20260709 12:20:32.373782 32648 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:32.378175 32613 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:32.353899 32461 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:32.365960 32549 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.393340 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.394550 25708 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:32.396610 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:32.399329 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 178276us
I20260709 12:20:32.401671 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:32.359982 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.378306 32739 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:32.446076 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.447137 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.448110 26338 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:32.443786 32461 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:32.463295 32752 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:32.464053 32752 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.465066 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.465992 26335 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:32.472257 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:32.474700 32655 tablet_copy_client.cc:538] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:32.475036 32764 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:32.475921 32764 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.478830 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:32.479050 32713 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.479822 25632 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.480014 32713 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:32.487390 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 86057.9us
I20260709 12:20:32.443948 32613 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.488840 32613 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.495723 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:32.496805 32734 log.cc:927] Time spent T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:20:32.498960 25712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57968 (request call id 3622) took 1254 ms. Trace:
I20260709 12:20:32.499418 25712 rpcz_store.cc:276] 0709 12:20:31.244396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.333700 (+ 89304us) service_pool.cc:224] Handling call
0709 12:20:31.334172 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.454953 (+120781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.454982 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.455556 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:20:31.543419 (+ 87863us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:31.543474 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.543579 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.403659 (+860080us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b18003f99a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28702a6 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1
0709 12:20:32.405673 (+  2014us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b18003f99a0. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a23f3 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:20:32.498459 (+ 92786us) raft_consensus.cc:1685] finished
0709 12:20:32.498517 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.498760 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48027,"raft.queue_time_us":203555,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15011,"spinlock_wait_cycles":23818496,"thread_start_us":164755,"threads_started":2,"wal-append.queue_time_us":117987}
I20260709 12:20:32.498231 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:32.504869 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.503813 32764 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.506606 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.508025 32575 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:32.508459 32575 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.508810 32575 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:32.503022 32623 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:32.511894 32416 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:32.522156 25705 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:32.522598 25705 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:32.523816 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:32.523593 25705 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.524598 25705 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:32.525381 32461 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.526113 32461 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.528561 32461 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.530566 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.532794 25637 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:32.534914 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:32.542855 32752 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.543092 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.546157 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.547752 25636 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:32.548911 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:32.549201 25786 raft_consensus.cc:1240] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:32.551493 25631 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:20:32.553028 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:32.553802 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.554672 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.554905 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:32.567999 32252 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:32.568271 32763 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.093s	user 0.001s	sys 0.001s
I20260709 12:20:32.568804 32252 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.569473 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:32.554692 25706 raft_consensus.cc:2368] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19.
I20260709 12:20:32.555778 32713 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.556273 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.556460 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.560596 25750 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:32.563609 32613 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.560153 25531 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 16 to 20, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 20 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:32.566866 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:32.579695 32252 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:32.580591 32762 log.cc:927] Time spent T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.116s	user 0.000s	sys 0.004s
I20260709 12:20:32.581090   303 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:32.581574   303 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:32.570402   304 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:20:32.581969   303 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.546157 26308 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:32.581990 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.592581 25786 raft_consensus.cc:3060] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.594866 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.595374 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:32.604063 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:32.601891 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 67436.1us
I20260709 12:20:32.613749 25676 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:32.610559 25631 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:32.620407 25632 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:32.627440 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:32.631335 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61999.3us
I20260709 12:20:32.616680 25752 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:32.634711 32752 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.641873 32752 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.645097 32752 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:32.645164 32709 raft_consensus.cc:2749] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:32.621024 25531 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:32.649830 25676 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:32.626721 25584 auto_rebalancer.cc:629] Failed to schedule move for tablet 2c9b915f0d524c19b87995309e7ea872: Illegal state: Replica 955233c139344131a847044f17f8d348 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:32.650086 32593 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:32.650871 25531 catalog_manager.cc:5697] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 12 to 19, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 19 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:32.621961 32545 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.660543 32545 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:32.653657 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.659706 25531 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 13 to 18, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 18 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:32.661221 25707 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19.
I20260709 12:20:32.663192 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.664407 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.607069 32567 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:32.665082 25677 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:32.668114 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:32.654376 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.674896   305 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:32.675403 32472 raft_consensus.cc:2804] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:32.676069 32472 raft_consensus.cc:697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:32.676540   310 log.cc:927] Time spent T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:20:32.677368 32472 consensus_queue.cc:237] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.678098 32545 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:32.678630 32545 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:32.679034 32545 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:32.675419   305 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:32.680205   305 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:32.683714 25603 leader_election.cc:400] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19.
I20260709 12:20:32.684073 25603 leader_election.cc:403] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:20:32.685420 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.686244 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:32.688321 25676 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:32.686780 32416 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19.
I20260709 12:20:32.688074 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:32.689718 25632 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.690881 32613 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:32.690639 32752 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:32.692539 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:32.654996 26341 raft_consensus.cc:1240] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Rejecting Update request from peer c7bf8f20701245bcbaa09cc663025f97 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:32.707998 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.704674 32549 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:32.708928 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.708187 25531 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 13 to 15, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 15 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:32.592294 32713 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:32.708205 25584 auto_rebalancer.cc:629] Failed to schedule move for tablet c2f7c369dcd245eeba8804ccb208c79a: Illegal state: Replica 0ae979356d694a09be4b0cd976ea354a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:20:32.606463 32478 consensus_meta.cc:287] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.012s
I20260709 12:20:32.710467 26337 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:32.616851 32462 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:32.686466   303 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.725232 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:32.705267 32752 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:32.622471 25530 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 306) took 51023 ms (client timeout 15000 ms). Trace:
I20260709 12:20:32.733613 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:32.743661 25786 raft_consensus.cc:2468] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:32.726272 32462 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:32.749248 25783 raft_consensus.cc:2368] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19.
W20260709 12:20:32.748571 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 202548us
I20260709 12:20:32.772037 25752 leader_election.cc:400] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19.
I20260709 12:20:32.772583 25752 leader_election.cc:403] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:32.774760 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:32.775408 25787 raft_consensus.cc:3060] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.724046 32713 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.781469 32713 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.735105 32752 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:32.797762 25677 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:32.800690 25531 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 13 to 18, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 18 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:32.730993 32478 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.771723 32462 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:32.809258 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:32.809661 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61385.4us
W20260709 12:20:32.809962 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 90170.8us
I20260709 12:20:32.821334 32756 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:32.821995 32766 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.822090 32756 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.822800 32766 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.824038 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:32.825968 26308 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:32.827134 32259 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.827680 32259 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.829969 32259 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.830811 32759 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:32.830999 32623 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:32.831838 32623 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.834192 26338 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:32.835155 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:32.837033 25632 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:32.832252 32761 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:32.853214 25787 raft_consensus.cc:2468] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:32.833384 32439 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:32.811550 32764 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.866400 25677 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:32.795099 32715 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.873493 25531 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:32.835520 32759 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.894641 32259 consensus_queue.cc:1059] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:32.896112 32545 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.896811 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:32.899267 32592 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.899569   306 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:32.901962 32715 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:32.903017 32731 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19.
I20260709 12:20:32.937777 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:32.938373 25706 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.921981 32545 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.924969 32752 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.927034   303 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:32.882611 32549 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:32.929966 32655 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:20:32.930370 25531 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 14 to 16, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:32.945652 32478 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:32.946372 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 131941us
I20260709 12:20:32.946954 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:32.947755 32592 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.948854 32764 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:32.964461 32766 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:32.981066 32439 raft_consensus.cc:3055] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:32.910077 32759 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:32.992267 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 128976us
I20260709 12:20:32.997907 32713 raft_consensus.cc:2804] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:33.012363 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:33.009331 25711 pending_rounds.cc:85] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Aborting all ops after (but not including) 11
I20260709 12:20:33.020406 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:32.962280 32575 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.026333 25531 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 19, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 19 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:33.022372 25711 pending_rounds.cc:107] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:20:33.037277 25711 raft_consensus.cc:2889] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:33.034825 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 170296us
I20260709 12:20:33.013139 32439 raft_consensus.cc:740] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:33.049360 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:33.045559 32709 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:33.055073 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58452.5us
I20260709 12:20:33.055963 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:33.060025 32752 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:33.065469 32252 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:33.064973 25531 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 14 to 17, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 17 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:33.066135 32252 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.067456 25531 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 15 to 17, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 17 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.069844 32715 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.070052 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:32.737821 25530 rpcz_store.cc:269] 0709 12:19:41.598472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.678829 (+ 80357us) service_pool.cc:224] Handling call
0709 12:20:32.622160 (+50943331us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:28.113740 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:28.322259 (+208519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:28.325615 (+  3356us) write_op.cc:435] Acquired schema lock
0709 12:20:28.330142 (+  4527us) tablet.cc:662] Decoding operations
0709 12:20:28.404374 (+ 74232us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:28.413470 (+  9096us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:28.448135 (+ 34665us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:28.462419 (+ 14284us) tablet.cc:701] Row locks acquired
0709 12:20:28.467884 (+  5465us) write_op.cc:260] PREPARE: finished
0709 12:20:28.472094 (+  4210us) write_op.cc:270] Start()
0709 12:20:28.472298 (+   204us) write_op.cc:276] Timestamp: P: 1783599628472018 usec, L: 0
0709 12:20:28.472330 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:20:28.473347 (+  1017us) log.cc:844] Serialized 541 byte log entry
0709 12:20:28.975076 (+501729us) op_driver.cc:464] REPLICATION: finished
0709 12:20:29.138669 (+163593us) write_op.cc:301] APPLY: starting
0709 12:20:29.138750 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:29.139782 (+  1032us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:29.139892 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:29.140757 (+   865us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:29.141035 (+   278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:29.141100 (+    65us) write_op.cc:312] APPLY: finished
0709 12:20:29.141276 (+   176us) log.cc:844] Serialized 25 byte log entry
0709 12:20:29.150730 (+  9454us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:29.150869 (+   139us) write_op.cc:454] Released schema lock
0709 12:20:29.150929 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:29.151327 (+   398us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:29.151360 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":811774,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":163289,"apply.run_cpu_time_us":3411,"apply.run_wall_time_us":12633,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8313,"num_ops":1,"prepare.queue_time_us":334085,"prepare.run_cpu_time_us":21960,"prepare.run_wall_time_us":519707,"raft.queue_time_us":273364,"raft.run_cpu_time_us":7490,"raft.run_wall_time_us":176337,"replication_time_us":502478,"spinlock_wait_cycles":30464,"thread_start_us":531932,"threads_started":4,"wal-append.queue_time_us":212741,"wal-append.run_cpu_time_us":17058,"wal-append.run_wall_time_us":1455801}]]}
I20260709 12:20:33.079335 32439 consensus_queue.cc:260] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.080736 32439 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:33.082093 32592 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:33.030294 32761 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.090523 26338 raft_consensus.cc:3060] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:33.090914 32761 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.093051 32761 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:33.096482 32462 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.098542   305 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.100159 32739 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:33.101275   305 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:33.103159 32575 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.025550 32648 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:33.104506 25676 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:33.105515 32743 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:33.105909 32764 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:33.029093 32713 raft_consensus.cc:697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:33.107746 32713 consensus_queue.cc:237] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.024387   306 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.113344   306 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.119290 25712 raft_consensus.cc:1240] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:33.069204 25531 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.086301 25706 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:33.129562 26338 raft_consensus.cc:2468] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
W20260709 12:20:33.133704 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 307830us
I20260709 12:20:33.137369 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.147176 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.149526 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.132589 32252 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:33.159219 25531 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 12, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 12 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:33.171905 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 311689us
I20260709 12:20:33.140117 32592 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.184541 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:33.183365 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:33.186643 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:33.187302 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:33.192646 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.190460 25784 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:20:33.191170 25531 catalog_manager.cc:5697] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 13, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 13 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.202536 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.203480 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:33.204017 25637 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:33.203953 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:33.168035 32462 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:33.211383 25706 raft_consensus.cc:1240] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:33.208281 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:33.213935 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 172719us
I20260709 12:20:33.202073 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:33.237053 32549 raft_consensus.cc:2764] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:20:33.214977 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:33.213783 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:33.242272 25601 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.244731 25782 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:33.213603 32752 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:33.209748 32743 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.245790 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.242178 25531 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 9 to 19, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 19 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:33.210425 32759 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.249109 32593 consensus_queue.cc:1059] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:33.252338 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:33.205119   306 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.236326 32731 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:33.259359 25531 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.207273 32592 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.261063 32549 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:20:33.271723 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 59056us
I20260709 12:20:33.214138 26338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:33.269799 25531 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.277344 25601 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.277941   317 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.278666   317 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.281019   317 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.276321 26338 raft_consensus.cc:606] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:33.269870 32743 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.288436   319 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:33.290138 32743 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:33.350430 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 47162.9us
I20260709 12:20:33.367396 32759 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:33.273841 32731 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.368736 32593 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.369518 32593 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.370687 32731 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:33.296479   319 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:33.373122 32592 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.373543 25637 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
W20260709 12:20:33.373832 26310 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:33.378846 25531 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:33.402869 25531 catalog_manager.cc:5697] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:33.402311 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 125024us
W20260709 12:20:33.407497 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55535.9us
W20260709 12:20:33.410918 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 194681us
I20260709 12:20:33.290783   306 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.419451 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:33.420646 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:33.420848 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 205959us
I20260709 12:20:33.423094 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.425851 26338 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
W20260709 12:20:33.423707 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 189437us
I20260709 12:20:33.424368 25752 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:33.405793 32592 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:33.438222 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:33.435381 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 191416us
I20260709 12:20:33.430178 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:33.472179 25636 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:33.447715 32613 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:33.442195 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.450784   305 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.417814 32743 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.447312 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:33.446756 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.441893 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:33.433498 25531 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 13 to 17, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 17 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.458702   333 consensus_queue.cc:1059] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:33.459858 32592 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.462458 32542 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:33.463485 32715 raft_consensus.cc:2804] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:33.464453 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.465754 32756 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.450987   321 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.502964   321 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.448307 32593 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:33.475864   313 log.cc:927] Time spent T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.456s	user 0.004s	sys 0.000s
I20260709 12:20:33.433127 32759 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.504081 26338 raft_consensus.cc:1240] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:33.504539 32759 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.505220   321 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.475044 32252 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:33.506873 32252 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.508985 32252 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:33.448261 32655 tablet_bootstrap.cc:700] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Time spent opening tablet: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:20:33.510799 32575 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.511528 32575 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.513952 32592 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.505264 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.516925 26335 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:33.495756 25531 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 14, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 14 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:33.521471 25531 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 15, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 15 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:33.524783 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:33.481707 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.483929 25782 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:33.478819   329 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:33.492308 32731 raft_consensus.cc:3055] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:33.526984 32731 raft_consensus.cc:740] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:33.492424   332 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.526767 32259 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:33.528313   332 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:33.489692 32542 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:33.494333 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 70067.7us
I20260709 12:20:33.502837 32743 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.531333 32542 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.532562 32542 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:33.532961 32743 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:33.538707 32715 raft_consensus.cc:697] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:33.538614   305 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.539909 32715 consensus_queue.cc:237] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.545181 26337 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:33.546123 32766 consensus_queue.cc:1059] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:33.547518 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.548077 32766 raft_consensus.cc:3055] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:33.548465 32766 raft_consensus.cc:740] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:33.549307 32766 consensus_queue.cc:260] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.552031 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.552393 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.553126 26339 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:33.555373   306 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.557161 32579 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:33.557289 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 92942.3us
I20260709 12:20:33.548107 25711 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:33.560194 26335 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:33.562081 32461 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.562695 32461 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:33.564960 32461 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:33.568066 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:33.568780 25782 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:33.568825 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.569386 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.570756   334 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:33.571063 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:33.572026 32764 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:33.573694 25638 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:33.573971 25632 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:33.575294 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.576974 25531 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:33.575312 25751 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.578922 32731 consensus_queue.cc:260] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.579847   306 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:33.580308 32731 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:33.552769 26340 raft_consensus.cc:2368] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:33.583240 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.584497 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:33.585168 25783 raft_consensus.cc:3060] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:33.586282 32759 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.587644 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:33.591701 32623 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.591956   306 consensus_queue.cc:1059] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:33.593078 32764 raft_consensus.cc:3055] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:33.593497 32764 raft_consensus.cc:740] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:33.593616   332 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.594595 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:33.596750 32575 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.594691 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:33.527025 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45413.6us
I20260709 12:20:33.597867   332 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:33.598682 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.599026 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:33.601845   334 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:33.602010 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.602259   334 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.602653   334 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:33.603381 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.604529 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.607103 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.610287 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.610967 25636 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:33.612463 25750 leader_election.cc:400] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:33.612749 25750 leader_election.cc:403] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:33.612998 32592 consensus_queue.cc:1059] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:33.614082 32592 raft_consensus.cc:3055] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:33.614401 32592 raft_consensus.cc:740] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:33.615368 32592 consensus_queue.cc:260] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.616150 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.481477 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:33.619571 26308 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:33.594466 32764 consensus_queue.cc:260] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.624801 26336 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 20: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:33.627199 25783 raft_consensus.cc:2468] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:33.629524 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:33.623433 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 142114us
I20260709 12:20:33.631768 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:33.634259 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.637027 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:33.632637 32743 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:33.636525 26337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:33.643322 26337 raft_consensus.cc:606] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:33.632390 32592 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:33.643774 26337 raft_consensus.cc:614] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:33.641489 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.634600 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.639607 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.637694 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:33.645730 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.638723 32549 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:33.647754 32761 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:33.658560 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:33.620582   332 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.643194 32743 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.656002 26339 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for term 20 because replica has last-logged OpId of term: 19 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 16.
I20260709 12:20:33.647812 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.632908 32766 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:33.669934 26307 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.634984 32623 raft_consensus.cc:2804] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:33.654691 32713 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:33.681802 26307 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.655463   305 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:33.658891 32761 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.674924   332 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:33.677218 32743 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:33.681910 32764 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:33.691828 25676 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:33.697742   328 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:33.698235 32648 raft_consensus.cc:993] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:33.684406 32462 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:33.702766 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:33.702977 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:33.709950 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:33.692749 32623 raft_consensus.cc:697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:33.720816 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 66917.6us
I20260709 12:20:33.658126 32709 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:20:33.668641   330 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:20:33.738256 25782 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:33.723599 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:33.693681 32761 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:33.690251 32575 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:33.734635 25778 raft_consensus.cc:2393] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 19: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:33.740367 25783 raft_consensus.cc:2368] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19.
I20260709 12:20:33.743978 25676 leader_election.cc:400] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19.
I20260709 12:20:33.740087 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:33.654415   320 consensus_queue.cc:1175] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7bf8f20701245bcbaa09cc663025f97" responder_term: 19 status { last_received { term: 19 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 19 index: 14 } }
I20260709 12:20:33.750802 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:33.733634 32759 raft_consensus.cc:1081] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:33.754381 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.754577 32623 consensus_queue.cc:237] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.757216 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.759856 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:33.763944 25637 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:20:33.765321 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:33.766916 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.766983 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:33.769164 32648 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:33.786001 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.746529 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:33.797878 25786 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
W20260709 12:20:33.809242 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 62801.8us
I20260709 12:20:33.812088 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:33.797530 25751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.23.217.65:41483 timed out after 1.588s (SENT) [suppressed 7 similar messages]
I20260709 12:20:33.816350 25778 raft_consensus.cc:1275] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:33.826851 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:33.840479 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.841471 32439 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:33.878922 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:33.880093 25778 raft_consensus.cc:2393] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 20: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:33.804225 32648 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.920090 32648 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:33.810549   330 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:20:33.813898 32472 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:33.924981 32472 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:33.925463 32472 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:33.932448   323 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:20:33.937955   332 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:33.941175   332 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:33.942906   334 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.945719   334 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:33.827622 32259 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:33.951265 25781 raft_consensus.cc:2393] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 20: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:20:33.953899 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 124594us
I20260709 12:20:33.957475 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:33.781940 32743 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:33.966614 25782 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:33.971865 26308 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:33.980046 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:34.025279 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:34.029961 25632 raft_consensus.cc:1240] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:34.030599 25637 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:33.987353 32575 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:34.029910 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 232410us
I20260709 12:20:34.034924 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.037053 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:34.037151 25629 raft_consensus.cc:2368] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19.
I20260709 12:20:34.026774   330 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:34.046904   330 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:34.049263   330 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.050521 32549 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:34.050971 32613 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:34.051090 32549 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:34.051731 32613 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:34.052768   331 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:34.053192   331 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:34.053579   331 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:34.053978 32613 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.053388 32549 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.050356 32713 consensus_queue.cc:1048] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:34.077350 25783 raft_consensus.cc:1275] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:34.077759 25780 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:34.105336 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.110989 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:34.111690 25783 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:34.109368 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:34.126825 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:34.115831 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:34.130548 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 43994.4us
I20260709 12:20:34.099241 32761 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:34.145605 32761 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.147212 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.138005 32752 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:34.149027 26339 raft_consensus.cc:1240] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:34.146265 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.141173 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:34.153770 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 82153.2us
I20260709 12:20:34.142654 25778 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:34.162060 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:34.155004 32655 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.109s	user 0.000s	sys 0.003s
I20260709 12:20:34.174458 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.178056 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.178220 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.180052 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:34.160081 32752 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:34.137140 32739 consensus_queue.cc:1048] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:20:34.210140 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.214584 32752 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:34.204865 32613 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:34.218114   345 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:20:34.233268 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.231108 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.232573 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:34.231755 32743 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:34.228173 32613 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:34.243294 32613 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:34.247189   348 log.cc:927] Time spent T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:20:34.249598 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.251986 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.254154 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.237632 32472 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:34.255477 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.256815 32472 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:34.257259   347 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:20:34.257629 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.259123 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.258754 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:34.264034 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64148.1us
I20260709 12:20:34.266083 25712 raft_consensus.cc:1240] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:34.271904 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.274044 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.275242 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.272092 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.272748 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.277266 25634 raft_consensus.cc:2368] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:34.283740   331 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:34.285034 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:34.298568 25711 raft_consensus.cc:1240] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:34.317636 32648 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:34.320037 32648 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.322214 25677 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:34.309427   362 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:34.323704   353 log.cc:927] Time spent T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:20:34.324376 32739 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:34.326517 32739 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:34.335791 32761 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:34.317997 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.352487 25601 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:34.359570 25709 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:34.360694 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:34.361465 25779 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 18: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:34.365430   305 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:34.366055   305 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:34.366598   362 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:34.368172 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:34.370850 25783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:34.371498 25783 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:34.371861 25783 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:34.374140 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:34.374951 26337 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 18: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:20:34.377583   360 log.cc:927] Time spent T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:20:34.368208   305 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.366708   330 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:34.382444   330 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:34.386729 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.394351 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:34.411054 26336 raft_consensus.cc:1275] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:34.422602 26342 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:20:34.423099 26341 raft_consensus.cc:1240] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:34.424230 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:34.425420 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:34.425593 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:34.426127 25779 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:34.443926   330 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.445909 32756 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:34.454031   331 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.467687 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.468506 32752 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:34.472116 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:34.482836 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:34.490232 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.492436 26307 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:34.520962 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.545578 25751 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:34.537431 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:34.548192 25783 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:34.554016 25787 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:34.554785 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:34.558300 25779 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:34.560421 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:34.520776 32713 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:34.536298   329 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:34.582103 25783 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:34.534015   327 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:34.582850   320 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:34.558830   328 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:20:34.571113 32764 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:34.564894 32593 consensus_queue.cc:1048] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:20:34.599112 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.600194 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:34.535521 32613 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:34.607959 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 47582.9us
I20260709 12:20:34.623170 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:34.624076 25708 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:34.573334 32478 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:20:34.643774 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 94975.7us
I20260709 12:20:34.650466 25630 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:20:34.651453 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 103791us
I20260709 12:20:34.652871   362 consensus_queue.cc:1059] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:34.654243   362 consensus_queue.cc:1059] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:34.656035   362 raft_consensus.cc:3055] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:34.656515   362 raft_consensus.cc:740] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:34.658922 32623 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.660307 32259 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:34.660329 32623 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:34.663887 32623 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:34.657737   362 consensus_queue.cc:260] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:34.666692   362 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:34.632442   347 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:20:34.610150   363 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:34.684634 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:34.749670   363 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:34.783828   371 log.cc:927] Time spent T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:20:34.631556 32759 consensus_queue.cc:1059] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:20:34.686321 25630 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:35.091017   363 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:34.818902 25604 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:34.817519 25708 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
W20260709 12:20:35.006206 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 419014us
W20260709 12:20:35.079098 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 420145us
I20260709 12:20:35.096796 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.099673   335 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:35.098410 25751 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:35.102082 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.085672 25601 leader_election.cc:400] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19.
I20260709 12:20:35.104384   334 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:35.104439 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:34.807132 32567 raft_consensus.cc:3055] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:35.107584 25751 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.087548 32613 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:35.091488 32752 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:35.102699   335 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:35.109848   335 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:35.108908 25751 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.111340   335 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:35.112957 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.113859 25783 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:35.106284 32567 raft_consensus.cc:740] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:35.112978 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.115595   335 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:35.116084 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.117055   335 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:35.117667 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:35.084946   370 log.cc:927] Time spent T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.245s	user 0.007s	sys 0.022s
I20260709 12:20:35.124390 32567 consensus_queue.cc:260] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:35.126096 32567 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:35.057225 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 453440us
I20260709 12:20:35.160604 26307 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:35.161980 32694 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:35.162200 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57899us
I20260709 12:20:35.163265 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:35.164423   317 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:35.164774 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:35.164831 32579 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:35.167224   317 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.167707   317 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:35.174139   329 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.174721   329 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.176234 32694 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.176887 32694 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:35.177807   329 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:35.178957 32694 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:35.179495 25752 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.180609 32715 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:20:35.179023 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 61450.2us
I20260709 12:20:35.181727 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.181686   334 raft_consensus.cc:2804] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:35.185544   334 raft_consensus.cc:697] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:35.186461   334 consensus_queue.cc:237] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.187565 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:35.188330 32416 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:35.189149 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:35.235693 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46620.9us
I20260709 12:20:35.236569   362 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:35.238116   369 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.148s	user 0.003s	sys 0.000s
I20260709 12:20:35.238336   362 consensus_queue.cc:1059] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:35.239185 32416 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:35.239530 32416 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:35.240448 32416 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.222015   380 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.241704 32416 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:35.225057 32579 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:35.226785 25708 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:35.231595 25706 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:35.206657   302 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:35.233517 32545 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.234061 25712 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:35.234182 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52561.3us
I20260709 12:20:35.244558 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.258750 25751 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:35.246474 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 97703.6us
I20260709 12:20:35.266635 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.269248 25751 leader_election.cc:400] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483) with higher term. Message: Invalid argument: T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20.
I20260709 12:20:35.270663 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:35.271256 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.273375 25783 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:35.252372 32752 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.277923 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:35.279332 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:35.284757 25787 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:35.276073 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 180599us
I20260709 12:20:35.265431 32766 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:35.288416 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.289777 25630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:35.290596 25630 raft_consensus.cc:606] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:35.291038 25630 raft_consensus.cc:614] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:35.267596   328 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.009s	sys 0.015s
I20260709 12:20:35.263291   362 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:35.294725   328 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.009s	sys 0.015s
I20260709 12:20:35.268507   335 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:35.302559 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.241715   380 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.310674 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.323197 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:35.327840 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.308480   385 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.296317   331 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.315918 32648 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:35.272174 32545 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.322306   374 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:35.322971 32713 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.377089 32713 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.379206 32713 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.381045 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.381830 25637 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:35.382543 32495 consensus_queue.cc:1059] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:35.384052   343 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:35.384624   343 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:35.318321   305 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:35.321689 32694 consensus_queue.cc:1059] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:20:35.370083   385 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.319250   362 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:20:35.374326 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:35.383854 32648 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.299201 32752 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.408795 32752 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.411962 32648 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:35.368151   380 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.417443 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:35.389864 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ac10fecbdc4482e93572bd7ff87319e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:35.402017   343 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.442581 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.443230 25708 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:35.385318   331 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.402808   305 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.446801 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.447335 25778 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:35.404722 32545 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.453975 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.406806 32694 raft_consensus.cc:3055] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:35.418401 26336 raft_consensus.cc:3055] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:35.421610 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:35.424209 32461 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:35.424925   362 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:35.469370 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:35.425848 32567 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:35.435994   320 raft_consensus.cc:3055] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:35.436303   385 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.437093 32764 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.402062 32462 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.438047   328 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.438088   335 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.460000 32694 raft_consensus.cc:740] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:35.450946 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.451589 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:35.437216   380 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.460160 26336 raft_consensus.cc:740] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:35.439510 32655 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:35.460981 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:35.498184 25630 raft_consensus.cc:2410] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for term 20 because replica has last-logged OpId of term: 19 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 16.
I20260709 12:20:35.499001 25676 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:35.467353 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:35.473665 25603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:35.447907 32648 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:35.452467   375 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.459942 32752 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:35.439733 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.467911 32495 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:35.473482   320 raft_consensus.cc:740] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:35.483932   335 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.480016 32764 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.478886   343 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:35.485494 32439 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:35.473088   368 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.526682 25711 raft_consensus.cc:1240] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:35.475503   334 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.448094   317 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.576779 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:35.581586 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 102068us
I20260709 12:20:35.523852 26336 consensus_queue.cc:260] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.586660 26336 raft_consensus.cc:3060] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:35.532684 32739 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:35.524600 32709 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:35.561373   302 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:35.482141 32462 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.597273 26339 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e13025a828f141378027526ae4f0b002"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:35.560374   377 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:20:35.597263 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 123776us
I20260709 12:20:35.601568 26339 raft_consensus.cc:606] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:35.605980 26339 raft_consensus.cc:614] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.592733 32593 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:35.537933   380 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.590554   334 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.622303 32593 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:35.622720   331 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:35.622934   388 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:20:35.579892 32439 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.599514 32567 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.624455 26339 raft_consensus.cc:1240] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:35.599766   302 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.624858 26338 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:35.523739   328 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.538362 32648 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.676559 32593 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:35.676756   302 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:35.677320 32648 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:35.609344 32694 consensus_queue.cc:260] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:35.639464 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 199861us
I20260709 12:20:35.604882   377 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:35.682235   377 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:35.683703 32462 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:35.611078 32715 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:35.685837   334 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.631249 32655 tablet_bootstrap.cc:492] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:20:35.686766 32694 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:35.687278 32439 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:35.687493 32655 ts_tablet_manager.cc:1403] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 2.758s	user 0.061s	sys 0.029s
I20260709 12:20:35.670440 32567 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.675004   328 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.683811   377 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:35.538264   375 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.704897   335 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.705508   305 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:35.597450   368 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.708226   317 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:35.708663 25785 raft_consensus.cc:1275] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:35.708947 26336 raft_consensus.cc:2468] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:35.732941 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.735666 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.740319 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.737236 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.743230 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.736438 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.740799 25711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:52030
I20260709 12:20:35.753477 25706 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 21: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:35.757774 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.752977 25709 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 20: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:35.763500 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.764709 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:35.765298 25632 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:35.767135 32764 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.767915   380 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.737528 32655 raft_consensus.cc:359] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:35.769609 32655 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:20:35.770077 25751 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:35.757714 25711 raft_consensus.cc:606] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:35.734630   356 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.787458   356 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.789557   397 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:35.791316 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.794026 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:35.795018 26336 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:35.775660 25711 raft_consensus.cc:614] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.795646 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:35.778306   303 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:35.778615 32655 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:35.797291 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.798323 25712 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
W20260709 12:20:35.754240   327 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:20:35.801141 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.802048 25780 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:35.790279   356 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:35.800231 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.806840 25783 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:35.807035 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:35.803045 25676 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:35.807690 26338 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:35.809612 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:35.810417 26343 raft_consensus.cc:2393] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 18: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:35.810699 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.811559 25778 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
I20260709 12:20:35.814826 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:35.835387 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 333031us
I20260709 12:20:35.777136 25632 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 21.
I20260709 12:20:35.805509 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.852975 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdf55909074ccd9e6b49679e14434d" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.793367 32623 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:35.855198 25601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:35.860154 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 60464.9us
I20260709 12:20:35.861513 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:35.861961 32472 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.862569   330 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:35.862633 32472 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.865134   330 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:35.858333 25787 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:35.866379 32472 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.870281 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.871233 25632 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:35.871308 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.871990 25712 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:35.872082 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.872752 25708 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:35.873023 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.873677 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.873881 25709 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 20: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:35.874303 25708 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:35.874384 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.875337 25632 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:35.877566 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.879937 32655 ts_tablet_manager.cc:1434] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.192s	user 0.007s	sys 0.002s
I20260709 12:20:35.884416 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.881526 32416 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:35.893736 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:35.892720 25676 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.907079   375 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.907606 26343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:35.908581 26343 raft_consensus.cc:606] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:35.782405   302 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.910702 25781 raft_consensus.cc:2393] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 22: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:35.912114   302 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.915621   368 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:35.920544 26308 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:35.920634 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 43617.6us
I20260709 12:20:35.924597 25752 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.926950 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.928313 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.932653 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.936273   380 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:35.936712   380 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.937355   380 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:35.929479 32416 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.933399   376 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:35.939003   376 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:35.939471   376 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:35.939263 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:35.940236 25711 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:35.940826   375 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:35.941313   375 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.941699   375 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:35.942531 32416 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.945351 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:35.946133 25711 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:35.946632   303 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:35.947104   303 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.947492   303 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:35.947522 25752 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.929888 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:20:35.950515 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:35.951228 25751 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:35.955452 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:35.958865   381 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:35.959336   381 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.959681   381 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:35.960866 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:35.961665 25780 raft_consensus.cc:3060] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:35.967056 32593 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:35.967562 32593 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.967880 32701 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:35.968019 32593 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:35.968354 32701 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:35.969345 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.969499 32701 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.971760 32567 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:35.978751   375 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:35.983031 32701 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:35.983042 25632 raft_consensus.cc:1240] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:35.986845 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.987654 25783 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 13.
I20260709 12:20:35.988758 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.991222 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:35.991870 25781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:35.992084   332 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:35.992673 25781 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:35.992777   385 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:35.995985   385 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:35.996017 26308 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:35.996729   385 raft_consensus.cc:3060] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:35.993306   332 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.998970   332 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:35.999917 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:36.001924 25710 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:36.003271   381 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.005393   381 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.008461   320 consensus_queue.cc:260] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.011592 32766 consensus_queue.cc:1059] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:20:36.014406   385 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.016664   385 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:36.017277 25711 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
I20260709 12:20:36.018594   332 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.020094 25752 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:36.020887   302 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:36.020912 25782 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:36.022234 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:36.023337 25705 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:20:35.970108 25784 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:35.995229 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:35.995103 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:36.022336 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:36.033923 25711 raft_consensus.cc:2393] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 20: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:35.990113 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:36.038642   373 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:36.039155   373 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:36.040086   373 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.041460   373 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:36.045281 25778 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:20:36.022315   302 raft_consensus.cc:2804] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:36.046845   302 raft_consensus.cc:697] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 22 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:36.047695   302 consensus_queue.cc:237] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.050932   377 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:36.051417   377 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:36.051649 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:36.051906   377 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:36.058442 25705 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:36.067749 25780 raft_consensus.cc:2468] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 21.
I20260709 12:20:36.070348 25704 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:35.991379   376 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.075765 25713 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Rejecting Update request from peer 0ae979356d694a09be4b0cd976ea354a for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:36.083638   380 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.084295   376 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.084868 25709 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:36.086006   380 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.099511 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:36.100132 25709 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:36.099453 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:36.100550 25630 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
W20260709 12:20:36.103410 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 308905us
I20260709 12:20:36.118849 25676 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:36.123766 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:36.124752 25784 raft_consensus.cc:2368] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19.
I20260709 12:20:36.126889   332 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.129699   368 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:20:36.134439 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 138500us
I20260709 12:20:36.101464   375 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.136826   363 raft_consensus.cc:2804] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:36.136952   320 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:36.137558   363 raft_consensus.cc:697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 21 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:36.137009 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:36.139181   362 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:36.139730   362 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.138821 32759 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.140677 32759 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.141873   362 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:36.143026 32759 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:36.145569 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:36.146358 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:36.146454 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.152521 26339 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:36.147014 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:36.148378 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.154701 26335 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:36.149158 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.151762 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.146836 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.155953 26336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.156672 26336 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:36.157754 25630 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:36.157686 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.081866 32593 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.151203   363 consensus_queue.cc:237] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.155668 26343 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:36.157065 25752 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:36.166286 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:36.161906 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595303b3057546cbbfda4b84f8df8912" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.167328 26341 raft_consensus.cc:3060] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:20:36.165114 32709 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:36.170380 32764 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:36.171036 32764 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:36.172057 32764 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.167826 25603 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.179958 26338 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 18 LEARNER]: Advancing to term 19
I20260709 12:20:36.184288 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.182929 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.185632 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.096223   303 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.186226 26344 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 16.
I20260709 12:20:36.188743 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.194659 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.194178 25786 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 19: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:36.197552 25708 raft_consensus.cc:2393] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 17: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:36.197599 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.205681 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-f80a8201dcaf4f2a9290c47595f9cf7b received from {username='slave'} at 127.0.0.1:60736
I20260709 12:20:36.206307 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-f80a8201dcaf4f2a9290c47595f9cf7b on tablet f80a8201dcaf4f2a9290c47595f9cf7b with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:20:36.206739   374 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:36.209774 25676 leader_election.cc:400] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19.
I20260709 12:20:36.194630 26337 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
I20260709 12:20:36.223644 25676 leader_election.cc:403] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:20:36.183843 32545 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:36.216940 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.225587 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.220566 26340 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 20: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:36.225790 26335 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:36.217768 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701d2ef98204ad0bedc9231fe829f69" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.226712 25676 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:36.227218 26339 raft_consensus.cc:3060] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:36.226714 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.228256 25709 raft_consensus.cc:2368] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19.
I20260709 12:20:36.228328 25676 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:36.232831 25782 raft_consensus.cc:2368] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 20. Current term is 21.
I20260709 12:20:36.245136 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:36.252468 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:36.253090 25633 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:20:36.252523 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:36.255086 25708 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:36.250313 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.264842 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.238915 32259 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:20:36.268476 26344 raft_consensus.cc:1275] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:36.252794   378 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.278728   378 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.281010   378 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:36.238955 32549 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:36.276089 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:36.233498 32709 raft_consensus.cc:2764] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:20:36.283857   385 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:36.284034 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:36.284834 25779 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:36.287698 32593 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.240579   381 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 18. Current term is 19.
I20260709 12:20:36.290238 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:36.290871   303 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.292049   319 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:36.292522   319 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.292896   319 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:36.268656 32478 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:36.268728   416 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:36.259544   387 consensus_queue.cc:1048] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:20:36.325301 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.326251 26337 raft_consensus.cc:3060] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:36.333128 26339 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:36.348264 25750 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:36.350183 25708 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:36.266441 26309 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:36.352459 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 86098.4us
I20260709 12:20:36.353291 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:36.353961 26335 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:36.322562   343 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:36.354846 25632 raft_consensus.cc:2468] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:36.356042 26309 leader_election.cc:400] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19.
I20260709 12:20:36.374543 26309 leader_election.cc:403] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:36.359720 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.379097 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a760df354a14284b004635d45d88a3e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.380069 25708 raft_consensus.cc:2393] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 21: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:36.393934 26337 raft_consensus.cc:2468] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:36.405647 25750 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:36.357571 32593 raft_consensus.cc:2804] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:36.357512 32648 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:36.358692   407 raft_consensus.cc:2764] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:20:36.376682   329 raft_consensus.cc:3060] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:36.356551   401 consensus_queue.cc:1048] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:20:36.418246   406 raft_consensus.cc:2804] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:36.372001   343 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.440936 26341 raft_consensus.cc:2468] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
I20260709 12:20:36.424690 32593 raft_consensus.cc:697] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:36.422369   356 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:20:36.442904   406 raft_consensus.cc:697] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:36.475279 26307 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.490746 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:36.504066 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82627.4us
I20260709 12:20:36.507417 25710 raft_consensus.cc:2393] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 19: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:36.505961 32593 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.518769   406 consensus_queue.cc:237] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.501292   368 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:36.512229   377 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.534919   377 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.535224 25633 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:36.535353   319 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:36.536264   422 log.cc:927] Time spent T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:20:36.538939   423 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:20:36.540890   343 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.549059 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.550560   356 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:36.554147 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.555258   319 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:36.557368 25751 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.561115 25603 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.562855 32739 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.563475 32739 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.571553 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.574352 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:36.574998 25787 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:36.580443 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:36.584514 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:36.588132   334 raft_consensus.cc:2804] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:36.588704   334 raft_consensus.cc:697] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:36.589636   334 consensus_queue.cc:237] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.600533 25630 raft_consensus.cc:2468] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:36.601280 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:36.601572 25676 leader_election.cc:400] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 20. Current term is 21.
I20260709 12:20:36.601931 25704 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:36.602028 25676 leader_election.cc:403] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:20:36.609979 25787 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:36.610975 32694 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:36.616875 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.623541 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.625178   329 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) with higher term. Message: Invalid argument: T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 for earlier term 18. Current term is 19.
I20260709 12:20:36.663389 25630 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
W20260709 12:20:36.626933 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76500.8us
I20260709 12:20:36.671602 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:36.672081 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:36.628825 32545 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061) with higher term. Message: Invalid argument: T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 for earlier term 20. Current term is 21.
I20260709 12:20:36.688505 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:36.673388   368 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.636415   416 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:20:36.705049 25601 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:36.708571 25704 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 20.
I20260709 12:20:36.673718   327 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:20:36.719251 26335 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:36.695577   343 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:36.693519   427 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:36.751911   431 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:36.756163 32739 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.777974 26343 raft_consensus.cc:2468] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 19.
I20260709 12:20:36.707198 32461 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:20:36.758582   428 raft_consensus.cc:3055] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:36.804842 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:36.806905 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
W20260709 12:20:36.805886 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 117180us
I20260709 12:20:36.813207 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:36.802922   303 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:36.732226   409 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:36.839354 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.842087 25706 raft_consensus.cc:1275] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:20:36.838443 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 107017us
I20260709 12:20:36.710866   368 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.803201   376 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:36.850627 26344 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:36.852065 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:36.811573   428 raft_consensus.cc:740] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:36.857620 32709 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:36.858122   397 consensus_queue.cc:1048] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:36.866226 32648 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:36.872619   430 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:20:36.873692   368 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:36.833766   303 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.875523   428 consensus_queue.cc:260] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.879407 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:36.848692   409 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 20 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:36.855458 32732 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.893431 25713 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:36.857054   376 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.898991 32732 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:36.912322   409 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.913966 32732 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.914744 25634 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:36.915536   303 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:36.903394   320 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:20:36.923180   376 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:36.933262 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:36.934437   329 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:36.934867   329 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.935228   329 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:36.948863 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:36.949692 26344 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 22: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:36.950400   320 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:36.950953   320 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.952920   320 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:36.955466 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.956254 25704 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 21: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:36.956969 25632 raft_consensus.cc:1240] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:36.949004 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:36.959543 25713 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:36.961944 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:36.962730 25634 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 22.
W20260709 12:20:36.966814 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 61724.9us
I20260709 12:20:36.964504 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:36.968173   368 consensus_queue.cc:1059] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:36.969815   372 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:36.970328   372 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:36.970742   437 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:36.972035   437 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:36.975154   437 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:36.975615   427 raft_consensus.cc:1064] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:20:36.979595   428 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:36.970762   372 raft_consensus.cc:3060] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:36.981715 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:36.989895 25634 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:36.995590   397 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:36.979516   427 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.010300 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:37.012411 25787 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 18.
I20260709 12:20:37.010617 32259 raft_consensus.cc:993] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:37.031893 25787 raft_consensus.cc:1275] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:20:37.043507 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:37.047024 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:37.047746 25778 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 18.
I20260709 12:20:37.050581 25750 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:37.054481 26337 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:37.064116 32593 raft_consensus.cc:2804] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:20:37.064595 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 98485us
I20260709 12:20:37.073330 25708 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:37.064821 32593 raft_consensus.cc:697] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:37.077802 32593 consensus_queue.cc:237] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.030587 32478 raft_consensus.cc:1081] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:37.080366 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:37.085984 25704 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:37.094197 25632 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:37.089946 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:37.080825 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:37.095841 25635 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:37.095170 25783 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:37.129236 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:37.130934 32549 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:37.131443 32549 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:37.131851 32549 raft_consensus.cc:3060] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:37.136973 25704 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:37.138684   431 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:37.116768   334 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
W20260709 12:20:37.145309 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 92627.6us
I20260709 12:20:37.112057   332 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:20:37.184130 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:37.185472 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:37.190033 25787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:37.190647 25787 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:37.191047 25787 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:37.182853   438 consensus_queue.cc:1048] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:20:37.201982 26343 raft_consensus.cc:1275] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:37.203303   303 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
W20260709 12:20:37.213984   447 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:20:37.185478 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:37.144923   329 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:37.236361   442 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:20:37.200651   332 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:37.198328   320 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:37.253259 25787 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.255995 25633 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:37.257017 25635 raft_consensus.cc:1275] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:37.214051 32549 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:37.266077 32549 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:37.267125   446 log.cc:927] Time spent T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:20:37.267858 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:37.230640   409 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:37.273839   409 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.274190 26338 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 19: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:37.259133   332 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:37.275856   332 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:37.252794 32766 consensus_queue.cc:1048] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:20:37.268257   329 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:37.283950 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:37.274224   409 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:37.289625 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:37.290452 25635 raft_consensus.cc:2393] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
I20260709 12:20:37.302440 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:37.303123 25786 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:37.303236 25778 raft_consensus.cc:2468] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 16.
I20260709 12:20:37.314332 25676 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:37.327751 25787 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:37.331504 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:37.332340 26344 raft_consensus.cc:3055] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:37.257790   320 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:37.332724 26344 raft_consensus.cc:740] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:37.347957 26344 consensus_queue.cc:260] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.355813 26344 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:37.328119   381 raft_consensus.cc:2804] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:37.361089 25786 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:37.362809 26307 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:37.349841   320 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:37.395080 25709 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:37.341663   397 consensus_queue.cc:1048] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:20:37.407845   372 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.346138 32593 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:20:37.381479   363 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:37.379820   381 raft_consensus.cc:697] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:37.455370   423 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:20:37.469661   454 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:20:37.474963   381 consensus_queue.cc:237] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:37.467955 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 148287us
I20260709 12:20:37.428709   357 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:20:37.461164   363 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
W20260709 12:20:37.508889 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 175456us
I20260709 12:20:37.512188 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:37.512766 25629 raft_consensus.cc:3060] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:37.523162   363 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.528410 25629 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
W20260709 12:20:37.537122 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 70637us
I20260709 12:20:37.547650   397 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:37.548527   397 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.629701 26335 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 19 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:37.615152   397 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:37.656144 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 92955.9us
I20260709 12:20:37.530848   320 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:37.759778 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:37.760605 25629 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
W20260709 12:20:37.761726 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 52571.5us
I20260709 12:20:37.761528 25751 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:37.752781   454 log.cc:927] Time spent T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 12:20:37.796370   458 log.cc:927] Time spent T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 12:20:37.819972 26344 raft_consensus.cc:2468] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
W20260709 12:20:37.799504   455 log.cc:927] Time spent T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
W20260709 12:20:37.832230 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 70777.3us
W20260709 12:20:37.835804   457 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
W20260709 12:20:37.836122   409 consensus_meta.cc:287] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 12:20:37.838246   375 raft_consensus.cc:2804] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:20:37.847600 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 87974.9us
W20260709 12:20:37.849848 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 286076us
I20260709 12:20:37.836593   409 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.782792   332 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:37.860955   375 raft_consensus.cc:697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:37.862171   375 consensus_queue.cc:237] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:37.862658   409 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:37.864279 25633 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 19: Already voted for candidate 0ae979356d694a09be4b0cd976ea354a in this term.
W20260709 12:20:37.866001 25634 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.004s
W20260709 12:20:37.866436 25634 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.000s	sys 0.004s
W20260709 12:20:37.853044   453 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
I20260709 12:20:37.876698 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:37.888607 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
W20260709 12:20:37.890552 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 181695us
I20260709 12:20:37.891139 25783 raft_consensus.cc:2393] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ae979356d694a09be4b0cd976ea354a in current term 19: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:37.892968 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:37.896951 25712 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 20: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:37.903517 25786 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:20:37.893185   456 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:20:37.949726   372 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:37.950728   320 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:37.532603 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:37.962103 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:37.963021 26343 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:37.970288 26342 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:20:37.987645 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60307.1us
I20260709 12:20:38.005651 25704 raft_consensus.cc:1275] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:38.006908   437 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:38.012194 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:38.023952   332 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:38.027460 25704 raft_consensus.cc:2393] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 21: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:38.044342 25750 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:38.066606 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:38.068691 25635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:38.069515 26338 raft_consensus.cc:2393] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 22: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:38.070683 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:38.072580 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:38.077831 25786 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:38.085372   411 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:38.088688 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:38.089605   410 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:38.091409   431 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:38.091894   431 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:38.092319   431 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:38.092976   376 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:38.080121 32472 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.099109 32472 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:38.101287 32472 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:38.091897   474 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:20:38.114399 25635 raft_consensus.cc:2468] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 22.
I20260709 12:20:38.114948 25638 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:20:38.120584 25629 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:38.121922 25677 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:38.123427 32549 raft_consensus.cc:2804] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:38.123951 32549 raft_consensus.cc:697] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:38.124899 32549 consensus_queue.cc:237] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:38.113839 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:38.128708 25531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 198) took 10686 ms. Trace:
I20260709 12:20:38.138854 25531 rpcz_store.cc:276] 0709 12:20:27.441980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.442231 (+   251us) service_pool.cc:224] Handling call
0709 12:20:38.128596 (+10686365us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:34.157488 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:34.301469 (+143981us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:34.303793 (+  2324us) write_op.cc:435] Acquired schema lock
0709 12:20:34.305376 (+  1583us) tablet.cc:662] Decoding operations
0709 12:20:34.784368 (+478992us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:34.803173 (+ 18805us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:34.803206 (+    33us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:20:34.805282 (+  2076us) tablet.cc:701] Row locks acquired
0709 12:20:34.805326 (+    44us) write_op.cc:260] PREPARE: finished
0709 12:20:34.805548 (+   222us) write_op.cc:270] Start()
0709 12:20:34.805728 (+   180us) write_op.cc:276] Timestamp: P: 1783599634805467 usec, L: 0
0709 12:20:34.805770 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:20:34.823943 (+ 18173us) log.cc:844] Serialized 9022 byte log entry
0709 12:20:35.642211 (+818268us) op_driver.cc:464] REPLICATION: finished
0709 12:20:35.938924 (+296713us) write_op.cc:301] APPLY: starting
0709 12:20:35.939008 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:36.214276 (+275268us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:36.214410 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:36.919144 (+704734us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:36.919818 (+   674us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 12:20:36.919894 (+    76us) write_op.cc:312] APPLY: finished
0709 12:20:36.920420 (+   526us) log.cc:844] Serialized 204 byte log entry
0709 12:20:36.994825 (+ 74405us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:37.006310 (+ 11485us) write_op.cc:454] Released schema lock
0709 12:20:37.022161 (+ 15851us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:37.117933 (+ 95772us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:37.128442 (+ 10509us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3367,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":273828,"apply.run_cpu_time_us":43871,"apply.run_wall_time_us":1118771,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100405,"num_ops":23,"prepare.queue_time_us":352370,"prepare.run_cpu_time_us":32766,"prepare.run_wall_time_us":1142930,"raft.queue_time_us":167493,"raft.run_cpu_time_us":35198,"raft.run_wall_time_us":326400,"replication_time_us":804562,"thread_start_us":737727,"threads_started":5,"wal-append.queue_time_us":333513,"wal-append.run_cpu_time_us":21039,"wal-append.run_wall_time_us":2295397}]]}
I20260709 12:20:38.148473 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:38.149266 25711 raft_consensus.cc:2393] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:20:38.132091 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 87822.4us
W20260709 12:20:38.153457 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 63235.2us
I20260709 12:20:38.155170 25634 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:38.156805 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:38.132503 25532 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.134338   376 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:20:38.134907 25637 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 13.
I20260709 12:20:38.157755 25786 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
I20260709 12:20:38.135852   372 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:38.172725 25676 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:38.171833   431 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:38.169127 32545 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.179843 32545 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:38.180289 25532 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.190940   334 consensus_queue.cc:1048] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:20:38.197831 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:38.194520 25532 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 16 to 18, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 18 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.206844 25751 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:38.199015   431 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:38.214893 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:38.219944 25532 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:38.223035 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50406.7us
W20260709 12:20:38.251803 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 45094.2us
I20260709 12:20:38.252665 25751 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:38.251909 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 60903.2us
I20260709 12:20:38.256363 26335 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:38.254154   372 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:38.265108 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:38.265812 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52068.2us
I20260709 12:20:38.268885 25706 raft_consensus.cc:2393] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 20: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:38.255143   320 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:38.259196   465 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:38.285979   368 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.286677   368 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:38.289811   465 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:38.290975   465 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:38.270924   372 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.292344   372 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:38.297546 25638 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:38.299432 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:38.315331 32545 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:38.315137 25532 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 13 to 16, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.307379   368 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:38.318313 25532 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 8 to 16, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 16 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.330721 25532 catalog_manager.cc:5697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 11 to 13, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 13 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.278288   320 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 21 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:38.349931   320 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:38.342908 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:38.355731 25637 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
W20260709 12:20:38.343751 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 67755.5us
I20260709 12:20:38.366887 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:38.369786   330 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:38.370299   330 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.370724   330 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:38.373708 25779 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:38.345757 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:38.379041   334 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:38.379755   334 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:38.344027 25629 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:38.381013   334 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:38.386510 26335 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:20:38.392210 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:38.398173   329 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:38.401717   497 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:20:38.412940   332 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:38.429119 25633 raft_consensus.cc:1275] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:20:38.433210   474 consensus_queue.cc:1048] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:38.467417 25532 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 15, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 15 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.477408 25778 raft_consensus.cc:1275] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:20:38.507428 25532 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.531862 25532 catalog_manager.cc:5697] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.506044   437 consensus_queue.cc:1048] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:20:38.573199 32593 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:38.604082 25532 catalog_manager.cc:5697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 4 to 15, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:38.599131   457 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:38.329772 32472 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:38.627543 32593 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:38.621836   330 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:38.666803 32593 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:38.667119 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:38.668061 26342 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 14.
I20260709 12:20:38.669508 25750 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:38.675352 25532 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 15, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:38.681245 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:38.681977 25629 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:38.677665 25532 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 7 to 9, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 9 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.717375 25629 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:38.718773 25532 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 15 to 18, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 18 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:38.733237 25532 catalog_manager.cc:5697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 16, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 16 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:38.734177 26342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
W20260709 12:20:38.738932 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 210860us
I20260709 12:20:38.738426 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348
W20260709 12:20:38.859771 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121479us
I20260709 12:20:38.734907 26342 raft_consensus.cc:606] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:38.719928   431 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:38.696259   372 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:38.905527 32623 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:38.957417 25532 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 9 to 13, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 13 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.898654 25784 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:38.902881   431 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:38.914561   492 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
W20260709 12:20:38.974509 32549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.020s	sys 0.039s
I20260709 12:20:39.020462 25532 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 9 to 11, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 11 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.980639 32623 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.945700   500 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:38.981171   431 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:39.093600 25532 catalog_manager.cc:5697] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:39.110579 32623 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:39.119077 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 218324us
W20260709 12:20:39.098116   503 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:20:39.108726 25532 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 8 to 18, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 18 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.096555   330 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:39.095714 32549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.022s	sys 0.043s
I20260709 12:20:39.106994   431 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.104753   500 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.192157   431 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:39.197918   465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.022s	sys 0.041s
I20260709 12:20:39.213418 25710 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 21. Current term is 22. Ops: []
W20260709 12:20:39.215919   503 log.cc:927] Time spent T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 12:20:39.226464   465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.023s	sys 0.042s
W20260709 12:20:39.247104 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 355858us
I20260709 12:20:39.274978 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:39.275734 25637 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:39.280112 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:39.280820 25633 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:39.273461   372 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:39.272292   438 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.304327 25637 raft_consensus.cc:2468] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 22.
I20260709 12:20:39.280158 32739 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.359203   471 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:39.360430 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59975.5us
I20260709 12:20:39.380806 26338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
W20260709 12:20:39.392836 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 460768us
I20260709 12:20:39.397992 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:39.380853 25677 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:39.414423 25634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51522 (request call id 2999) took 1186 ms. Trace:
I20260709 12:20:39.414844 25634 rpcz_store.cc:276] 0709 12:20:38.192283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:38.192440 (+   157us) service_pool.cc:224] Handling call
0709 12:20:38.192773 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:38.213056 (+ 20283us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:20:38.260518 (+ 47462us) log.cc:844] Serialized 14 byte log entry
0709 12:20:38.583698 (+323180us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:38.612203 (+ 28505us) log.cc:844] Serialized 26 byte log entry
0709 12:20:38.612370 (+   167us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:20:38.612417 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:38.612506 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:39.378173 (+765667us) raft_consensus.cc:1685] finished
0709 12:20:39.378207 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:39.378366 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33994,"prepare.queue_time_us":46819,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":59553,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":21327,"thread_start_us":311746,"threads_started":1,"wal-append.queue_time_us":455935}
I20260709 12:20:39.427258 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:39.443149 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:39.424341   519 raft_consensus.cc:2804] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:39.397317 32739 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.447463 26335 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:39.449488 32739 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:39.411895   471 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.456308   471 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:39.451417 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.457623 25637 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:39.463855   519 raft_consensus.cc:697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:39.465484   500 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:39.465309   519 consensus_queue.cc:237] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:39.469306 32623 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.472565 26342 raft_consensus.cc:2368] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
W20260709 12:20:39.485036 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 50684.4us
I20260709 12:20:39.425895   479 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.379362   438 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:39.493698 32701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.129s	user 0.031s	sys 0.037s
I20260709 12:20:39.510466 32623 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:39.535799 26344 raft_consensus.cc:1275] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:20:39.537451 26338 raft_consensus.cc:606] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:39.537839 26338 raft_consensus.cc:614] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:39.518698 32701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.032s	sys 0.037s
I20260709 12:20:39.570567   521 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.571339   521 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.573923   521 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:39.499354   479 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.591156 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:39.599275 25633 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:20:39.599659 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:39.603626 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:39.604578 26342 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:39.602649 25677 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
W20260709 12:20:39.587071 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 168559us
I20260709 12:20:39.566435   520 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.618299 25712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:39.627018 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 213241us
I20260709 12:20:39.608006   441 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.655711   474 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:39.627902   489 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:39.656447   524 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.661600   331 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:20:39.667675 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49249.8us
I20260709 12:20:39.688896 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:39.689806 25637 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:39.667171   467 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:39.731729   467 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.734144   467 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:39.670467   441 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.769510   441 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:39.770049   441 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:39.704622 25750 leader_election.cc:400] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:20:39.770797 25750 leader_election.cc:403] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:20:39.771279 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 66831.8us
I20260709 12:20:39.706256 32545 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.772722 32545 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.774559   465 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.774961 32545 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:39.775096   441 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.775203   465 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:39.706827 32259 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.776806 32259 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.777899   441 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:39.778909 32259 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:39.773355   467 raft_consensus.cc:3060] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:39.708962   480 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.780912   480 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.783007   480 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:39.698122   427 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.698712 32739 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:39.784487   427 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.784885 32739 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.632653   520 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:39.711544 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46079.4us
I20260709 12:20:39.787663   520 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:39.735992   479 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:39.805621 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 153630us
I20260709 12:20:39.737587 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.739701 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:39.681131   489 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:39.743928 26338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:39.807511 26338 raft_consensus.cc:606] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:39.808156 26338 raft_consensus.cc:614] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:39.749615 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.682229   524 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:39.760310   532 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:20:39.684180   331 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.688060   490 raft_consensus.cc:3055] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:39.767086 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.840612 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.781176   465 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:39.812618 25712 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:39.811470 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:39.699927   438 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:39.818286 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:39.832401 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:39.842293 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.826779 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:39.830521 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 90915.3us
I20260709 12:20:39.840731 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.851832 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.855163 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:39.851963 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.859869 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.862023 25632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.828462 25637 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:39.863862 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:39.865183 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60646.7us
I20260709 12:20:39.876689 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:39.877589 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:39.805155   363 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.884419 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:39.885776 25635 raft_consensus.cc:1240] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:39.893090 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:39.894949 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:39.895107 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:39.880143 26308 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:39.896675 25631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:39.806083   480 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:39.805732   433 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:39.896852 25784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50966 (request call id 2719) took 1040 ms. Trace:
I20260709 12:20:39.895164 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:39.903553 25636 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:39.908186 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:39.918550 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:39.858211   479 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:39.903175 25784 rpcz_store.cc:276] 0709 12:20:38.856430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:38.856670 (+   240us) service_pool.cc:224] Handling call
0709 12:20:38.898183 (+ 41513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:38.898461 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:20:39.475082 (+576621us) log.cc:844] Serialized 14 byte log entry
0709 12:20:39.896350 (+421268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:39.896429 (+    79us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:39.896501 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:39.896601 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:39.896772 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22162,"thread_start_us":398584,"threads_started":1}
W20260709 12:20:39.946874 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46287.1us
W20260709 12:20:39.948129 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44889us
I20260709 12:20:39.851529   490 raft_consensus.cc:740] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:39.896490 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:39.948606 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:39.952018 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:39.960322 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:39.961184 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 64770us
I20260709 12:20:39.900825   477 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:39.948181 32623 raft_consensus.cc:2804] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:39.978858 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:39.953409   479 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.951648 32259 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:39.896461 32472 consensus_queue.cc:1048] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:20:39.971195   438 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:39.922734   363 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.986181   479 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:40.033377   331 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.029124 32259 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.037035 32259 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:39.949451   480 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:39.964090 32739 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.024701 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:39.975412   520 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:40.015067 32623 raft_consensus.cc:697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:40.002863   538 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:39.949342   433 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:40.067107 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.047609 26343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:40.070345 26343 raft_consensus.cc:606] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:40.010732   489 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:40.029014   385 raft_consensus.cc:493] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:39.987063   477 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:40.094645 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 137518us
I20260709 12:20:40.112414 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
W20260709 12:20:40.093811 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 112589us
I20260709 12:20:40.127866 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:40.076731   363 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:40.107944   504 log.cc:927] Time spent T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:20:40.078469   433 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:40.164151 25676 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:40.140573 26307 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:40.140264 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 153752us
I20260709 12:20:40.168916 25750 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:40.084714   524 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:40.273587 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69169.1us
I20260709 12:20:40.072350   520 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.038513   490 consensus_queue.cc:260] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:40.166792   518 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.296706   320 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:40.298969   553 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:40.116300   385 raft_consensus.cc:515] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:40.321795   542 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.262s	user 0.000s	sys 0.000s
W20260709 12:20:40.322662   546 log.cc:927] Time spent T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:20:40.084298 32623 consensus_queue.cc:237] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:40.335758   547 log.cc:927] Time spent T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:20:40.339123   500 raft_consensus.cc:993] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
W20260709 12:20:40.344190 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 180140us
I20260709 12:20:40.357488 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:40.199043   363 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.118005   479 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.285161   522 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:40.200146   433 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:40.347604   320 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:40.362288   441 raft_consensus.cc:1081] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:40.362145   553 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:40.391968   320 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:40.384116   433 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.411414   433 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.413115   385 leader_election.cc:290] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.422715 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:40.429839   427 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.431613 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.434044 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.440291   357 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:40.442049 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:40.462842 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:40.464507 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:40.454686 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.372016   480 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.477202 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907e0d1516b4c78a2a977e29402bff9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:40.477464 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:40.512297 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 80810.8us
I20260709 12:20:40.513206 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.514418 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.515772 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.518458 25706 raft_consensus.cc:1240] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:40.521745 32259 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:40.522181   527 consensus_queue.cc:1059] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:40.523303   479 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.524374   527 raft_consensus.cc:3055] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:40.524799   527 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:40.526206 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 68066.1us
I20260709 12:20:40.527213   527 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:40.528460   527 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:40.531924   520 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:40.532114   477 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:40.535701 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.536212 25677 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:40.536821 32259 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:40.537498 32259 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:20:40.543396   467 consensus_meta.cc:287] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Time spent flushing consensus metadata: real 0.764s	user 0.006s	sys 0.006s
I20260709 12:20:40.543835   467 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023) with higher term. Message: Invalid argument: T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 for earlier term 14. Current term is 15.
I20260709 12:20:40.545475   433 consensus_queue.cc:1059] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:40.547338 25631 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:40.550140 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.516925   471 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:40.563447 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.563833 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.563525 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31ccbf41d24e4275a7f921db24b940d4" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.486263   357 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:40.575534 25752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.011s	sys 0.035s
W20260709 12:20:40.575963 25752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.011s	sys 0.035s
W20260709 12:20:40.576970 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 516339us
I20260709 12:20:40.556180 32545 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:40.580878 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:40.587007 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.590695 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.592033 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.595106 25778 raft_consensus.cc:2468] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:40.597030 25633 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:40.603202 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.619530 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.620666 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.621984 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.614779   525 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:40.613894   566 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:20:40.571033   471 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:40.640241 25752 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.639828 25676 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:40.643440 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 107813us
I20260709 12:20:40.652508 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.657799 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:40.658452 25630 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:40.665295 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.677438 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.677915 25782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:40.706877 25782 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:40.707420 25782 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:40.707844 25752 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.657020 32472 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:40.710065 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 44840.9us
I20260709 12:20:40.714334 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
I20260709 12:20:40.644351   334 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
W20260709 12:20:40.715391 25750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.714584   521 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.721197   471 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.651377   479 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:20:40.739982 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.739851 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.715210   428 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:40.710569   539 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:40.745146 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 74579.6us
I20260709 12:20:40.749699 25783 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:20:40.749963 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.750133 25783 pending_rounds.cc:85] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 8
I20260709 12:20:40.751441 25783 pending_rounds.cc:107] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:20:40.751711 25783 raft_consensus.cc:2889] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:40.748839 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78091.9us
I20260709 12:20:40.761420 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.744057   431 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:40.746523   433 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:40.772617 26309 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.780025 25676 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:40.760579   357 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:40.784003   521 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:40.794723 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:40.798960   576 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:40.800045   582 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:40.810904 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.758631   539 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.814597   539 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.790395   517 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:20:40.820641 25676 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:40.821864 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:40.822708   519 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.823150   489 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.823803   490 raft_consensus.cc:3060] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:40.801740 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:40.830828 26337 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:40.831700 25603 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.835323   411 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:40.837553 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:40.838852 25706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57966 (request call id 3360) took 1029 ms. Trace:
I20260709 12:20:40.839344 25706 rpcz_store.cc:276] 0709 12:20:39.809362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:39.809491 (+   129us) service_pool.cc:224] Handling call
0709 12:20:39.809740 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:40.534867 (+725127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:40.838609 (+303742us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400147b54. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598fe01 00007f29b598e83e 00007f29ba953419 00007f29b3ecabd4 00007f29b3ecab62 00007f29b3ecaa91 00007f29b3ecaa0c 00007f29b3ec9cc2 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:40.838786 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622407936}
I20260709 12:20:40.839848   357 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:40.853246 26337 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
I20260709 12:20:40.854565 25676 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 955233c139344131a847044f17f8d348
I20260709 12:20:40.856300 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.856525 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.857780 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.867527 25782 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.880666   582 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:40.881419   582 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:40.887744 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:40.892264 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.892604 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97aeee11fff4d52bb8252149577f56e" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:40.893409 26342 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:40.894873 25631 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:20:40.897655   562 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:40.900688 25636 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 12:20:40.907056   578 log.cc:927] Time spent T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:20:40.908929 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.914057 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:40.915355 25779 raft_consensus.cc:1240] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Rejecting Update request from peer 10d0c7349085463595aedaf7c071e742 for earlier term 17. Current term is 18. Ops: []
W20260709 12:20:40.918674 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 48805.3us
I20260709 12:20:40.924206 25751 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.927030   465 consensus_queue.cc:1059] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:40.927268   334 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.928052   465 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:40.928064 32648 raft_consensus.cc:993] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:40.928632 26307 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:40.929287 32648 raft_consensus.cc:1081] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:40.930114 25634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:40.930704 25634 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:40.930815 25750 leader_election.cc:304] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.931984   525 raft_consensus.cc:2749] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:40.928772   465 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:40.937345 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 45175us
I20260709 12:20:40.941007 25782 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:40.941744   520 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:40.937811 25634 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:40.945538 25751 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:40.952242   491 consensus_queue.cc:1059] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:40.882512   582 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.954537   520 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:40.955094   520 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.955785   491 raft_consensus.cc:3055] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:40.956156   491 raft_consensus.cc:740] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:40.956655 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:40.957376 25779 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:40.957050   491 consensus_queue.cc:260] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.957762 25779 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:40.957931   520 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:40.937785   465 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.959604   465 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:40.966917 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:40.969194 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:40.975960 26307 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:40.977898 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:40.958339   491 raft_consensus.cc:3060] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:40.953768   477 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:40.991931 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:40.986826 32648 raft_consensus.cc:2749] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.997315   320 raft_consensus.cc:2804] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:41.003334   320 raft_consensus.cc:697] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:41.004328   320 consensus_queue.cc:237] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:40.974246 25779 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:41.030504 25779 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:41.031489 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:41.032156 26344 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:41.032485 26344 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:41.041501 26344 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:41.042678 26344 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:41.042368 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
W20260709 12:20:41.048388 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 45901.3us
I20260709 12:20:41.052304 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:41.052883 25636 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:41.060024   564 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:41.088930 25636 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:41.112540 25786 raft_consensus.cc:1275] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:20:41.113174 25786 pending_rounds.cc:85] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 12
I20260709 12:20:41.113390 25786 pending_rounds.cc:107] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:20:41.113657 25786 raft_consensus.cc:2889] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:41.125972 32739 consensus_queue.cc:1048] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:41.232141 26344 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
W20260709 12:20:41.236758 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 159679us
I20260709 12:20:41.242772 26340 raft_consensus.cc:2468] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 18.
I20260709 12:20:41.242928 25712 raft_consensus.cc:1240] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:20:41.244606 25637 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:20:41.248406 25751 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:41.262631   385 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:41.263295   385 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:41.264354   385 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:41.278798   521 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:20:41.280222 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52007.3us
I20260709 12:20:41.295061 25784 raft_consensus.cc:1275] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:20:41.299733 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49798.1us
I20260709 12:20:41.319795   428 consensus_queue.cc:1048] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:41.308933 25531 catalog_manager.cc:5697] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:41.347995 25532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 301) took 35254 ms (client timeout 15000 ms). Trace:
I20260709 12:20:41.331141 25531 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 17 to 18, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 18 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.393349 25782 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:20:41.394032 25782 pending_rounds.cc:85] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Aborting all ops after (but not including) 12
I20260709 12:20:41.394373 25782 pending_rounds.cc:107] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:20:41.396781 25531 catalog_manager.cc:5697] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:41.394768 25782 raft_consensus.cc:2889] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:41.400782 25531 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:41.348522 25532 rpcz_store.cc:269] 0709 12:20:06.167991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.168191 (+   200us) service_pool.cc:224] Handling call
0709 12:20:41.347908 (+35179717us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:39.722838 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:39.723465 (+   627us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:39.723504 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:20:39.723523 (+    19us) tablet.cc:662] Decoding operations
0709 12:20:39.725054 (+  1531us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:39.725122 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:39.725153 (+    31us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:20:39.727509 (+  2356us) tablet.cc:701] Row locks acquired
0709 12:20:39.727532 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:20:39.727705 (+   173us) write_op.cc:270] Start()
0709 12:20:39.727838 (+   133us) write_op.cc:276] Timestamp: P: 1783599639727648 usec, L: 0
0709 12:20:39.727857 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:20:39.728655 (+   798us) log.cc:844] Serialized 7521 byte log entry
0709 12:20:40.712766 (+984111us) op_driver.cc:464] REPLICATION: finished
0709 12:20:40.873056 (+160290us) write_op.cc:301] APPLY: starting
0709 12:20:40.873142 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:40.908985 (+ 35843us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:40.909085 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:41.090198 (+181113us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:41.090472 (+   274us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 12:20:41.090539 (+    67us) write_op.cc:312] APPLY: finished
0709 12:20:41.090907 (+   368us) log.cc:844] Serialized 172 byte log entry
0709 12:20:41.091698 (+   791us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:41.091892 (+   194us) write_op.cc:454] Released schema lock
0709 12:20:41.091944 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:41.234549 (+142605us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:41.234578 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":221393,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":157900,"apply.run_cpu_time_us":26918,"apply.run_wall_time_us":361434,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":314306,"prepare.run_cpu_time_us":29515,"prepare.run_wall_time_us":266691,"raft.queue_time_us":298608,"raft.run_cpu_time_us":15017,"raft.run_wall_time_us":115295,"replication_time_us":946584,"thread_start_us":732927,"threads_started":4,"wal-append.queue_time_us":328400}]]}
I20260709 12:20:41.411511 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet ef4fd24cfa184aa8a8a1de8f32296e6a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:41.407357 25531 catalog_manager.cc:5697] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 18 to 20, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 20 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.442184 25531 catalog_manager.cc:5697] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 18 to 21, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 21 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:41.444856 25531 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:41.448143 25531 catalog_manager.cc:5697] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 15 to 19, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 19 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.464123 25531 catalog_manager.cc:5697] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.466365 25531 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 13, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 13 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.468509 25531 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 12 to 14, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 14 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.479099 25531 catalog_manager.cc:5697] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.459123   519 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:20:41.481737 25531 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 12 to 20, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 20 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.485312 25531 catalog_manager.cc:5697] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 15, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 15 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.488762 25531 catalog_manager.cc:5697] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 15 to 20, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 20 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.492280 25531 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 12, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 12 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.495432 25531 catalog_manager.cc:5697] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.510732 25631 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:41.498010 25531 catalog_manager.cc:5697] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:41.524515 26340 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:41.526295 25531 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.528004 25531 catalog_manager.cc:5697] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 20 to 22, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 22 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.529491 25531 catalog_manager.cc:5697] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 16 to 17, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 17 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:41.533897   594 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:41.535112   527 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:41.543370 25531 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.560712 25531 catalog_manager.cc:5697] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.564833 25531 catalog_manager.cc:5697] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.539005   510 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:20:41.566828 25531 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index -1 to 14, term changed from 14 to 19, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66), VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 19 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:41.652504 25630 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:41.654275   539 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:41.657452 26338 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:41.692845 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet ef4fd24cfa184aa8a8a1de8f32296e6a (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:41.687474   553 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 12:20:41.799708   597 log.cc:927] Time spent T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 12:20:41.806794 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 179545us
I20260709 12:20:41.826164 26344 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:41.884500 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:41.902822 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 71608.8us
I20260709 12:20:41.884032   557 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:20:41.949743 25631 raft_consensus.cc:1275] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:41.984689   566 consensus_queue.cc:1048] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:42.054215 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 97528us
W20260709 12:20:42.324322 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 59576.7us
W20260709 12:20:42.363332 25738 heartbeater.cc:646] Failed to heartbeat to 127.23.217.126:36567 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.23.217.126:36567 timed out after 15.000s (SENT)
I20260709 12:20:42.764324 25709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:52030
I20260709 12:20:42.809795 25709 raft_consensus.cc:606] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:42.810545 25709 raft_consensus.cc:614] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:42.833997 26338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:42.834676 26338 raft_consensus.cc:606] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:42.835045 26338 raft_consensus.cc:614] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:42.839334 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:42.840093 26340 raft_consensus.cc:606] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:42.840545 26340 raft_consensus.cc:614] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:42.843482 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e13025a828f141378027526ae4f0b002"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:42.844291 26340 raft_consensus.cc:606] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:42.849270 26340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:57550
I20260709 12:20:42.850034 26340 raft_consensus.cc:606] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:42.850665 26340 raft_consensus.cc:614] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:42.852032 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:42.856707 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:42.858034 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:42.864506 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 47922.9us
I20260709 12:20:42.887238   428 raft_consensus.cc:993] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:42.887845   428 raft_consensus.cc:1081] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:42.911665 25710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:42.912377 25710 raft_consensus.cc:606] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Received request to transfer leadership
I20260709 12:20:42.924912 25710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:42.925560 25710 raft_consensus.cc:606] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Received request to transfer leadership
W20260709 12:20:42.926878 25522 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for f80a8201dcaf4f2a9290c47595f9cf7b already in progress [suppressed 2 similar messages]
I20260709 12:20:42.957445 32739 raft_consensus.cc:993] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:42.958397   581 raft_consensus.cc:1081] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:42.958065 25536 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 16 to 20, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 20 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.960644 25536 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.965139 25536 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:42.966813 25536 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.968955 25536 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 16 to 20, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 20 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.971113 25536 catalog_manager.cc:5697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 8 to 11, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 11 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.975006 25531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 199) took 2037 ms. Trace:
I20260709 12:20:42.975309 26340 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56156
I20260709 12:20:42.978034 26340 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:42.978799 26340 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:42.975425 25531 rpcz_store.cc:276] 0709 12:20:40.937660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.937875 (+   215us) service_pool.cc:224] Handling call
0709 12:20:42.974708 (+2036833us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:41.929034 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:41.973940 (+ 44906us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:41.973985 (+    45us) write_op.cc:435] Acquired schema lock
0709 12:20:41.974010 (+    25us) tablet.cc:662] Decoding operations
0709 12:20:41.975840 (+  1830us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:41.975929 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:41.975955 (+    26us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:20:41.978197 (+  2242us) tablet.cc:701] Row locks acquired
0709 12:20:41.978219 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:20:41.978413 (+   194us) write_op.cc:270] Start()
0709 12:20:41.998338 (+ 19925us) write_op.cc:276] Timestamp: P: 1783599641978345 usec, L: 0
0709 12:20:42.003449 (+  5111us) op_driver.cc:348] REPLICATION: starting
0709 12:20:42.165137 (+161688us) log.cc:844] Serialized 9910 byte log entry
0709 12:20:42.424315 (+259178us) op_driver.cc:464] REPLICATION: finished
0709 12:20:42.437532 (+ 13217us) write_op.cc:301] APPLY: starting
0709 12:20:42.437629 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:42.457664 (+ 20035us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:42.457837 (+   173us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:42.565830 (+107993us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:42.571264 (+  5434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 12:20:42.571380 (+   116us) write_op.cc:312] APPLY: finished
0709 12:20:42.620999 (+ 49619us) log.cc:844] Serialized 220 byte log entry
0709 12:20:42.654135 (+ 33136us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:42.724232 (+ 70097us) write_op.cc:454] Released schema lock
0709 12:20:42.724338 (+   106us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:42.836443 (+112105us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:42.836503 (+    60us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":32077,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":12895,"apply.run_cpu_time_us":40251,"apply.run_wall_time_us":330896,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108777,"num_ops":25,"prepare.queue_time_us":273736,"prepare.run_cpu_time_us":24202,"prepare.run_wall_time_us":338641,"raft.queue_time_us":154693,"raft.run_cpu_time_us":2080,"raft.run_wall_time_us":8559,"replication_time_us":420679,"thread_start_us":330513,"threads_started":4,"wal-append.queue_time_us":97934}]]}
W20260709 12:20:42.983160 25522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 56426.8us
I20260709 12:20:42.985746 25536 catalog_manager.cc:5697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 13 to 15, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 15 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.987989 25536 catalog_manager.cc:5697] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 16 to 19, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 19 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.989729 25779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e13025a828f141378027526ae4f0b002"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50194
I20260709 12:20:42.990377 25779 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:42.990924 25779 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:43.001483 25536 catalog_manager.cc:5697] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:43.004140 25536 catalog_manager.cc:5697] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 11 to 16, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 16 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.010387 25536 catalog_manager.cc:5697] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:43.008756 25779 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:43.016099 25779 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:43.016412 25710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:43.012593 25536 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 11 to 17, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 17 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.017068 25710 raft_consensus.cc:606] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Received request to transfer leadership
W20260709 12:20:43.037775 25584 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:20:43.038331 25584 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for f80a8201dcaf4f2a9290c47595f9cf7b already in progress
W20260709 12:20:43.038648 25584 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for f80a8201dcaf4f2a9290c47595f9cf7b already in progress
I20260709 12:20:43.019049 25536 catalog_manager.cc:5697] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 13 to 15, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 15 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:43.017793 26340 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:43.040753 25536 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 15 to 18, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 18 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.043737 25536 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.045892 25536 catalog_manager.cc:5697] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:43.047900 25536 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 7 to 19, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 19 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.050091 25536 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 18 to 21, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 21 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.053290 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:43.054263 26338 raft_consensus.cc:3055] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:43.054597 26338 raft_consensus.cc:740] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:43.056277 25633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:43.055495 26338 consensus_queue.cc:260] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:43.056852 25633 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:43.057495 26338 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:43.064786 25633 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:43.067683 25710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:43.068353 25710 raft_consensus.cc:3055] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:43.068643 25710 raft_consensus.cc:740] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:43.067658 26340 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:43.084151 26338 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:43.073741 25710 consensus_queue.cc:260] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:43.103379 25710 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:43.105017 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:43.105792 25786 raft_consensus.cc:3060] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:43.110339 25750 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:43.117806 25710 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:43.118690   474 raft_consensus.cc:2804] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:43.130260 25786 raft_consensus.cc:2468] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:43.129820 26309 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:43.138470   611 raft_consensus.cc:2804] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:43.131389   474 raft_consensus.cc:697] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:43.139158   611 raft_consensus.cc:697] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:43.140214   474 consensus_queue.cc:237] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:43.140393   611 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:43.165371 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:43.166006 25630 raft_consensus.cc:3060] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:43.190093 25630 raft_consensus.cc:2468] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:43.297793 25531 catalog_manager.cc:5697] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 17 to 18, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 18 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:43.299690 25531 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 19 to 21, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 21 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:43.309995 25531 catalog_manager.cc:5697] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.318687 25531 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 17 to 18, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 18 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:43.422356 25536 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 306) took 12419 ms (client timeout 15000 ms). Trace:
W20260709 12:20:43.426573 25522 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8a1f6623a51a47759e59269cfa63be4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer c7bf8f20701245bcbaa09cc663025f97 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 21 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:43.422921 25536 rpcz_store.cc:269] 0709 12:20:31.003207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.003520 (+   313us) service_pool.cc:224] Handling call
0709 12:20:43.422231 (+12418711us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:43.174768 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:43.175502 (+   734us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:43.175553 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:20:43.175570 (+    17us) tablet.cc:662] Decoding operations
0709 12:20:43.177322 (+  1752us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:43.177428 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:43.177454 (+    26us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:20:43.178963 (+  1509us) tablet.cc:701] Row locks acquired
0709 12:20:43.179018 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:20:43.179205 (+   187us) write_op.cc:270] Start()
0709 12:20:43.179370 (+   165us) write_op.cc:276] Timestamp: P: 1783599643179140 usec, L: 0
0709 12:20:43.179394 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:20:43.195874 (+ 16480us) log.cc:844] Serialized 7196 byte log entry
0709 12:20:43.246877 (+ 51003us) op_driver.cc:464] REPLICATION: finished
0709 12:20:43.247273 (+   396us) write_op.cc:301] APPLY: starting
0709 12:20:43.248162 (+   889us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:43.273578 (+ 25416us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:43.273709 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:43.285762 (+ 12053us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:43.286090 (+   328us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:20:43.286140 (+    50us) write_op.cc:312] APPLY: finished
0709 12:20:43.286603 (+   463us) log.cc:844] Serialized 164 byte log entry
0709 12:20:43.286810 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:43.286992 (+   182us) write_op.cc:454] Released schema lock
0709 12:20:43.287060 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:43.287488 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:43.287523 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":490640,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":166,"apply.run_cpu_time_us":22685,"apply.run_wall_time_us":40152,"cfile_cache_hit":72,"cfile_cache_hit_bytes":75348,"num_ops":18,"prepare.queue_time_us":102872,"prepare.run_cpu_time_us":5872,"prepare.run_wall_time_us":43859,"replication_time_us":67322,"thread_start_us":32103,"threads_started":1}]]}
I20260709 12:20:43.491276 25531 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 19 to 20, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.552322 25633 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:43.555584   474 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:43.580094 25528 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 9 to 12, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 12 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.611893 25710 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:20:43.618265 25535 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 308) took 14091 ms (client timeout 15000 ms). Trace:
I20260709 12:20:43.619588   520 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:43.631713 25535 rpcz_store.cc:269] 0709 12:20:29.526691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.526933 (+   242us) service_pool.cc:224] Handling call
0709 12:20:43.618013 (+14091080us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":191403,"reported_tablets":48}
I20260709 12:20:43.695926 25636 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:43.699540   611 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:43.717576 25786 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:43.723701   520 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:20:43.754263 25786 consensus_queue.cc:237] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:43.795825 26342 raft_consensus.cc:1275] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:43.802743 25706 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:43.805600   474 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:43.812961   527 consensus_queue.cc:1048] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:20:43.831693 26342 raft_consensus.cc:1275] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:43.851307   474 consensus_queue.cc:1048] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:20:43.861449   649 log.cc:927] Time spent T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
W20260709 12:20:43.875277 25528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 305) took 34295 ms (client timeout 15000 ms). Trace:
W20260709 12:20:43.881119 25528 rpcz_store.cc:269] 0709 12:20:09.584452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.591574 (+  7122us) service_pool.cc:224] Handling call
0709 12:20:09.925175 (+333601us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b5400080610. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b286ab71 00007f29b286b0b2 00007f29bb7e4759 00007f29bb77ca76 00007f29bb74dd58 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc
0709 12:20:43.875162 (+33949987us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:43.635068 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:43.635703 (+   635us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:43.635748 (+    45us) write_op.cc:435] Acquired schema lock
0709 12:20:43.635762 (+    14us) tablet.cc:662] Decoding operations
0709 12:20:43.636173 (+   411us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:43.636406 (+   233us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:43.636459 (+    53us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:43.636711 (+   252us) tablet.cc:701] Row locks acquired
0709 12:20:43.636733 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:20:43.637361 (+   628us) write_op.cc:270] Start()
0709 12:20:43.637537 (+   176us) write_op.cc:276] Timestamp: P: 1783599643636874 usec, L: 0
0709 12:20:43.637565 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:20:43.638416 (+   851us) log.cc:844] Serialized 541 byte log entry
0709 12:20:43.653122 (+ 14706us) op_driver.cc:464] REPLICATION: finished
0709 12:20:43.653572 (+   450us) write_op.cc:301] APPLY: starting
0709 12:20:43.653674 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:43.669594 (+ 15920us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:43.670049 (+   455us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:43.671107 (+  1058us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:43.671329 (+   222us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:43.671391 (+    62us) write_op.cc:312] APPLY: finished
0709 12:20:43.671651 (+   260us) log.cc:844] Serialized 25 byte log entry
0709 12:20:43.671856 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:43.671953 (+    97us) write_op.cc:454] Released schema lock
0709 12:20:43.672009 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:43.672482 (+   473us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:43.672517 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":146702,"reported_tablets":48,"spinlock_wait_cycles":47934208,"child_traces":[["op",{"apply.queue_time_us":145,"apply.run_cpu_time_us":4326,"apply.run_wall_time_us":18869,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4186,"num_ops":1,"prepare.queue_time_us":12583,"prepare.run_cpu_time_us":3347,"prepare.run_wall_time_us":6232,"replication_time_us":15323}]]}
I20260709 12:20:43.947790   557 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 3 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:43.951587 25706 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 3 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:44.011683 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 89284.6us
I20260709 12:20:44.012418 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:44.034785 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7701d2ef98204ad0bedc9231fe829f69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:44.037180 26342 raft_consensus.cc:2955] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 3 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:44.063529 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 53492.1us
I20260709 12:20:44.063766 25535 catalog_manager.cc:5697] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 19 to 20, leader changed from 10d0c7349085463595aedaf7c071e742 (127.23.217.68) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 20 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:44.086447 25529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 307) took 37110 ms (client timeout 15000 ms). Trace:
W20260709 12:20:44.087018 25529 rpcz_store.cc:269] 0709 12:20:06.975729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.976049 (+   320us) service_pool.cc:224] Handling call
0709 12:20:07.775832 (+799783us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:44.086337 (+36310505us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2558462,"reported_tablets":48,"spinlock_wait_cycles":10184064}
I20260709 12:20:44.135695 25535 catalog_manager.cc:5697] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 3 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 17 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:44.165304 26342 consensus_queue.cc:237] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.168437 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 7701d2ef98204ad0bedc9231fe829f69 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:56416
I20260709 12:20:44.191095 25784 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:44.193892 25636 raft_consensus.cc:1275] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:44.194796   660 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:44.215531   660 consensus_queue.cc:1048] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:44.243382   662 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Committing config change with OpId 20.16: config changed from index 14 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:44.263403 25636 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 14 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:44.271554 25630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:44.272226 25630 raft_consensus.cc:606] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:44.272552 25630 raft_consensus.cc:614] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:44.273979 25534 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 305) took 84484 ms (client timeout 15000 ms). Trace:
W20260709 12:20:44.274363 25534 rpcz_store.cc:269] 0709 12:19:19.786753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.790195 (+  3442us) service_pool.cc:224] Handling call
0709 12:19:20.049800 (+259605us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f9090 00007f29b5997f3a 00007f29bb74de9c 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6
0709 12:20:44.270653 (+84220853us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":48,"spinlock_wait_cycles":297143168}
I20260709 12:20:44.283010 25784 raft_consensus.cc:2955] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 14 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:44.284780 25533 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 14, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 14 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.288609 25633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:44.289211 25633 raft_consensus.cc:606] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:44.315872 25630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:44.316506 25630 raft_consensus.cc:606] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:44.318126 25630 raft_consensus.cc:614] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:44.330181   664 tablet_replica.cc:333] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:44.334213   664 raft_consensus.cc:2243] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:44.335165   664 raft_consensus.cc:2272] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:44.338845 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f80a8201dcaf4f2a9290c47595f9cf7b with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:44.341387 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:44.350410 25636 raft_consensus.cc:606] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:20:44.352385 25534 catalog_manager.cc:5697] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 14 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.360054   664 ts_tablet_manager.cc:1916] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:44.385944   528 raft_consensus.cc:993] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:20:44.386723 32701 raft_consensus.cc:1081] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:20:44.377153 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:20:44.388679 25710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:20:44.389295 25710 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:44.389657 25710 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:44.387702 25784 raft_consensus.cc:606] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:20:44.409360 25636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:44.410102 25636 raft_consensus.cc:606] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:44.417253 32701 raft_consensus.cc:993] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:44.418038   625 raft_consensus.cc:1081] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:44.421427 26338 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56146
I20260709 12:20:44.422142 26338 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:44.422614 26338 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:44.410526 25636 raft_consensus.cc:614] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:44.451543 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:20:44.553926   664 ts_tablet_manager.cc:1929] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.11
I20260709 12:20:44.557387   664 log.cc:1199] T 7701d2ef98204ad0bedc9231fe829f69 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/7701d2ef98204ad0bedc9231fe829f69
I20260709 12:20:44.599375   539 raft_consensus.cc:993] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:44.600046   539 raft_consensus.cc:1081] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:44.604686 26343 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56168
I20260709 12:20:44.605227 26343 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:44.605547 26343 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:44.564670 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 7701d2ef98204ad0bedc9231fe829f69 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
W20260709 12:20:44.613056 25521 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 48575.8us
I20260709 12:20:44.615442 26343 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.621433 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:44.622253 25636 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:44.636974 26343 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:44.638186 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:44.641022 25784 raft_consensus.cc:3055] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:44.641338 25784 raft_consensus.cc:740] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:44.642211 25784 consensus_queue.cc:260] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.643385 25784 raft_consensus.cc:3060] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:44.654440 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:44.656654 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:44.657922 25784 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:44.663630 26338 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.665716 26307 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:44.666671 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:44.667132   662 raft_consensus.cc:2804] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:44.667943   662 raft_consensus.cc:697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:44.668064 26338 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:44.669109   662 consensus_queue.cc:237] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.667312 25630 raft_consensus.cc:3055] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:44.671312 25630 raft_consensus.cc:740] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:44.672534 25630 consensus_queue.cc:260] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.673384 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:44.673913 25630 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:44.674145 25704 raft_consensus.cc:3060] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:44.663722 25636 raft_consensus.cc:2468] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:44.704016 25710 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.708487 25630 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:44.712296 25532 catalog_manager.cc:5697] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 20 to 22, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 22 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.717440 26308 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
I20260709 12:20:44.719105   660 raft_consensus.cc:2804] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:44.719722   660 raft_consensus.cc:697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:44.720912   660 consensus_queue.cc:237] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:44.724601 25533 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 302) took 17333 ms (client timeout 15000 ms). Trace:
I20260709 12:20:44.729072 25704 raft_consensus.cc:2468] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:44.740078 25710 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:44.741492 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:44.742436 26343 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:44.726310 25532 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
I20260709 12:20:44.746402 25532 catalog_manager.cc:5697] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 17 to 19, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 19 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.750231 25532 catalog_manager.cc:5697] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 15 to 16, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.752374 25532 catalog_manager.cc:5697] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.756753 25532 catalog_manager.cc:5697] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:44.726603 25533 rpcz_store.cc:269] 0709 12:20:27.390892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.392835 (+  1943us) service_pool.cc:224] Handling call
0709 12:20:44.309470 (+16916635us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:44.309501 (+    31us) write_op.cc:454] Released schema lock
0709 12:20:44.724485 (+414984us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:44.293155 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:44.293686 (+   531us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:44.293762 (+    76us) write_op.cc:435] Acquired schema lock
0709 12:20:44.293777 (+    15us) tablet.cc:662] Decoding operations
0709 12:20:44.294133 (+   356us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:44.294188 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:44.294214 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:44.294394 (+   180us) tablet.cc:701] Row locks acquired
0709 12:20:44.294407 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:20:44.294575 (+   168us) write_op.cc:270] Start()
0709 12:20:44.294687 (+   112us) write_op.cc:276] Timestamp: P: 1783599644294528 usec, L: 0
0709 12:20:44.294705 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:20:44.295328 (+   623us) log.cc:844] Serialized 541 byte log entry
0709 12:20:44.298823 (+  3495us) op_driver.cc:464] REPLICATION: finished
0709 12:20:44.300341 (+  1518us) write_op.cc:301] APPLY: starting
0709 12:20:44.300421 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:44.301266 (+   845us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:44.301368 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:44.302131 (+   763us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:44.302297 (+   166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:44.302345 (+    48us) write_op.cc:312] APPLY: finished
0709 12:20:44.302538 (+   193us) log.cc:844] Serialized 25 byte log entry
0709 12:20:44.302696 (+   158us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:44.302783 (+    87us) write_op.cc:454] Released schema lock
0709 12:20:44.302823 (+    40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":1279,"apply.run_cpu_time_us":2781,"apply.run_wall_time_us":2781,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4186,"num_ops":1,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":2431,"prepare.run_wall_time_us":2431,"replication_time_us":3973}]]}
I20260709 12:20:44.766340 26343 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
I20260709 12:20:44.762245 25532 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 14, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 14 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.773890 25533 catalog_manager.cc:5697] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 18 to 19, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 19 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.781602 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:44.782362 25637 raft_consensus.cc:3055] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:44.782702 25637 raft_consensus.cc:740] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:44.783465 25637 consensus_queue.cc:260] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.785161 25637 raft_consensus.cc:3060] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:44.800345 25637 raft_consensus.cc:2468] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 15.
W20260709 12:20:44.804582 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 69695.4us
I20260709 12:20:44.803545 25532 catalog_manager.cc:5697] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 19 to 20, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 20 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } }
W20260709 12:20:44.809612 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 46561.2us
I20260709 12:20:44.820372 25676 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:44.821384   667 raft_consensus.cc:2804] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:44.861138 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet f80a8201dcaf4f2a9290c47595f9cf7b with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:44.822161   667 raft_consensus.cc:697] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:44.903676   667 consensus_queue.cc:237] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:44.893468 25637 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:44.983775 25704 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:44.985483 25782 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:44.985688   674 tablet_replica.cc:333] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:20:44.986965 25601 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c9b915f0d524c19b87995309e7ea872. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:44.987874   663 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:45.003435   625 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:45.004506   674 raft_consensus.cc:2243] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:45.005281   674 raft_consensus.cc:2272] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:45.003435 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 94164.4us
I20260709 12:20:45.016017 25533 catalog_manager.cc:5697] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 13 to 14, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 14 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:45.022432 25522 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8a1f6623a51a47759e59269cfa63be4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer c7bf8f20701245bcbaa09cc663025f97 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 21 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:45.036339   674 ts_tablet_manager.cc:1916] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:45.092398 25782 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:20:45.095189 25532 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42082 (request call id 303) took 2588 ms. Trace:
I20260709 12:20:45.094121 32701 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.097976   657 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:45.096334 25532 rpcz_store.cc:276] 0709 12:20:42.501707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:42.502298 (+   591us) service_pool.cc:224] Handling call
0709 12:20:42.798578 (+296280us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b54000805b8. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb74de43 00007f29bb84c04e 00007f29b9d0d2a4 00007f29b9d0d232 00007f29b9d0d161 00007f29b9d0d0dc 00007f29b9d0c392 00007f29b3cca48c 00007f29b3cc98c6 00007f29b3cc926f
0709 12:20:45.089524 (+2290946us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:44.841124 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:44.841971 (+   847us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:44.842014 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:20:44.842037 (+    23us) tablet.cc:662] Decoding operations
0709 12:20:44.843249 (+  1212us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:44.843324 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:44.843368 (+    44us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:20:44.844196 (+   828us) tablet.cc:701] Row locks acquired
0709 12:20:44.844209 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:20:44.844382 (+   173us) write_op.cc:270] Start()
0709 12:20:44.844513 (+   131us) write_op.cc:276] Timestamp: P: 1783599644844307 usec, L: 0
0709 12:20:44.844541 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:20:44.874718 (+ 30177us) log.cc:844] Serialized 3258 byte log entry
0709 12:20:44.878256 (+  3538us) op_driver.cc:464] REPLICATION: finished
0709 12:20:44.942402 (+ 64146us) write_op.cc:301] APPLY: starting
0709 12:20:44.946588 (+  4186us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:44.975850 (+ 29262us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:44.975948 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:44.992956 (+ 17008us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:44.993261 (+   305us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:20:44.993322 (+    61us) write_op.cc:312] APPLY: finished
0709 12:20:45.010153 (+ 16831us) log.cc:844] Serialized 81 byte log entry
0709 12:20:45.010411 (+   258us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:45.012422 (+  2011us) write_op.cc:454] Released schema lock
0709 12:20:45.012509 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:45.013146 (+   637us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:45.013177 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":117493,"reported_tablets":49,"spinlock_wait_cycles":14746368,"child_traces":[["op",{"apply.queue_time_us":48010,"apply.run_cpu_time_us":12589,"apply.run_wall_time_us":73638,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33488,"num_ops":8,"prepare.queue_time_us":2343,"prepare.run_cpu_time_us":4939,"prepare.run_wall_time_us":39198,"replication_time_us":33550,"thread_start_us":61921,"threads_started":1}]]}
I20260709 12:20:45.113375 25704 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.118160 25784 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.168292   674 ts_tablet_manager.cc:1929] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.15
I20260709 12:20:45.178182 25706 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:45.201182   674 log.cc:1199] T f80a8201dcaf4f2a9290c47595f9cf7b P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/f80a8201dcaf4f2a9290c47595f9cf7b
I20260709 12:20:45.204905 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet f80a8201dcaf4f2a9290c47595f9cf7b (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:45.220974 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 7701d2ef98204ad0bedc9231fe829f69 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:56416
I20260709 12:20:45.226397 25786 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:20:45.227849 25676 consensus_peers.cc:597] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c2fd18ad19c4394a69e6f9d6aa778e1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:45.229102   673 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:45.229552 25637 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:45.238999   673 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:45.250849 25634 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:20:45.252203   657 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:45.256992 25630 raft_consensus.cc:1275] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:20:45.258594   662 consensus_queue.cc:1048] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:45.280637 25706 raft_consensus.cc:1275] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:20:45.282382 25531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42066 (request call id 307) took 1684 ms. Trace:
I20260709 12:20:45.282845 25531 rpcz_store.cc:276] 0709 12:20:43.597680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:43.598649 (+   969us) service_pool.cc:224] Handling call
0709 12:20:45.282324 (+1683675us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":28}
I20260709 12:20:45.285470 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 7701d2ef98204ad0bedc9231fe829f69 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:45.288388   670 consensus_queue.cc:1048] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:45.303571 25631 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:45.320189   667 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:45.346302   677 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index -1 to 12, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.348074 25786 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index -1 to 12, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.362514 25637 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index -1 to 12, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.400714 25706 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:45.418151 25786 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:20:45.418952 26343 raft_consensus.cc:1275] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:45.420353   677 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:20:45.423664 25676 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12eaa1f2655a4cc7bde6bd409ccdf780. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:45.447994   677 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.449731 25782 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.457583 25637 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:20:45.478832   610 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:45.478528 25533 catalog_manager.cc:5697] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 14 to 15, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.485441   696 consensus_queue.cc:1048] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:20:45.531615 25704 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:45.518668 25531 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 16 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.548202   697 ts_tablet_manager.cc:933] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:20:45.616020   697 tablet_copy_client.cc:323] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:20:45.630844 25532 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 12, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.633013 25532 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 9, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.637708 25636 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.661602 25528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 309) took 1719 ms. Trace:
I20260709 12:20:45.662107 25528 rpcz_store.cc:276] 0709 12:20:43.942462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:43.942688 (+   226us) service_pool.cc:224] Handling call
0709 12:20:45.661533 (+1718845us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77527,"reported_tablets":36}
I20260709 12:20:45.667886 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet 2c9b915f0d524c19b87995309e7ea872 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:20:45.668557 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet 2c9b915f0d524c19b87995309e7ea872 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-2c9b915f0d524c19b87995309e7ea872
I20260709 12:20:45.694689 25784 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:20:45.695274 25633 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:20:45.696976   610 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:45.699254   667 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:20:45.707954 25676 consensus_peers.cc:597] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7fee45cced0b4ce18dea86254386fee0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:45.710588 25648 tablet_copy_source_session.cc:215] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:45.729986   697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9b915f0d524c19b87995309e7ea872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:45.772887   705 ts_tablet_manager.cc:933] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:45.792965 25633 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.797370   705 tablet_copy_client.cc:323] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:20:45.803407 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:20:45.804057 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-0c2fd18ad19c4394a69e6f9d6aa778e1
I20260709 12:20:45.819726 25784 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.789023   610 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.855553   697 tablet_copy_client.cc:806] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:45.856340   697 tablet_copy_client.cc:670] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:45.867358 25723 tablet_copy_source_session.cc:215] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:45.886974   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:45.898206 25528 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 11, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:45.951824   707 ts_tablet_manager.cc:933] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:45.952121   697 tablet_copy_client.cc:538] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:45.974272 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:46.000396   705 tablet_copy_client.cc:806] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:46.001082   705 tablet_copy_client.cc:670] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:46.015799   705 tablet_copy_client.cc:538] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:46.032042   697 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
W20260709 12:20:46.024407 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 99891.3us
W20260709 12:20:46.057893 25522 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8a1f6623a51a47759e59269cfa63be4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer c7bf8f20701245bcbaa09cc663025f97 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 21 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:46.078702   707 tablet_copy_client.cc:323] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:20:46.080724 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 12eaa1f2655a4cc7bde6bd409ccdf780 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:20:46.081346 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 12eaa1f2655a4cc7bde6bd409ccdf780 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-12eaa1f2655a4cc7bde6bd409ccdf780
I20260709 12:20:46.091342   705 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:20:46.125738 25723 tablet_copy_source_session.cc:215] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:46.138768   707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12eaa1f2655a4cc7bde6bd409ccdf780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:46.177019 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:20:46.177681 25784 raft_consensus.cc:606] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:20:46.180039   711 ts_tablet_manager.cc:933] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:46.182359 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:20:46.182963 25784 raft_consensus.cc:606] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:46.185892 25704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:52030
I20260709 12:20:46.186566 25704 raft_consensus.cc:606] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:46.189579 25784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:20:46.190176 25784 raft_consensus.cc:606] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:46.198019 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:46.200541   707 tablet_copy_client.cc:806] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:46.201231   707 tablet_copy_client.cc:670] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:46.200474 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:46.203868 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:46.226816   527 raft_consensus.cc:993] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:20:46.227432   539 raft_consensus.cc:1081] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:20:46.227537   707 tablet_copy_client.cc:538] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:46.229965 25637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60051e5723b2429582407076a003096c"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:20:46.236510 25637 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:46.236933 25637 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:46.238885   711 tablet_copy_client.cc:323] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:20:46.240607 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet 7fee45cced0b4ce18dea86254386fee0 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:20:46.241230 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet 7fee45cced0b4ce18dea86254386fee0 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-7fee45cced0b4ce18dea86254386fee0
I20260709 12:20:46.244735 25637 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:46.260231 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:46.260850 26342 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:46.259445 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:46.262733 25704 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:46.260354 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:46.263438 25784 raft_consensus.cc:3055] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:46.263759 25784 raft_consensus.cc:740] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:46.258101 25637 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:46.277976 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
I20260709 12:20:46.266685 25784 consensus_queue.cc:260] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:46.256065 25723 tablet_copy_source_session.cc:215] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:46.290282 26342 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
I20260709 12:20:46.295042 25784 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:46.305236 25603 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:46.307117   528 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:46.307796   528 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:46.309880   528 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:46.315070   711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fee45cced0b4ce18dea86254386fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:46.324801 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 21.
I20260709 12:20:46.336259   707 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:20:46.390437   697 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:46.404954   697 tablet_bootstrap.cc:492] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:20:46.406260   697 ts_tablet_manager.cc:1403] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.374s	user 0.038s	sys 0.008s
I20260709 12:20:46.433403   711 tablet_copy_client.cc:806] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:46.434777   711 tablet_copy_client.cc:670] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:46.485944   697 raft_consensus.cc:359] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:46.487095   697 raft_consensus.cc:740] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:20:46.499722   697 consensus_queue.cc:260] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:46.504438   697 ts_tablet_manager.cc:1434] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.098s	user 0.011s	sys 0.000s
I20260709 12:20:46.475706 25528 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 20 to 21, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 21 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.513343   711 tablet_copy_client.cc:538] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:46.528070 26338 raft_consensus.cc:1217] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.10->[16.11-16.11]   Dedup: 16.11->[]
I20260709 12:20:46.567605   716 raft_consensus.cc:993] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:46.568267   716 raft_consensus.cc:1081] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
W20260709 12:20:46.580452 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 53521.4us
I20260709 12:20:46.598263 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-2c9b915f0d524c19b87995309e7ea872 received from {username='slave'} at 127.0.0.1:60736
I20260709 12:20:46.598963 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-2c9b915f0d524c19b87995309e7ea872 on tablet 2c9b915f0d524c19b87995309e7ea872 with peer 10d0c7349085463595aedaf7c071e742
W20260709 12:20:46.620648 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 76163.6us
W20260709 12:20:46.625499 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 87434.3us
W20260709 12:20:46.674561 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 55808.7us
W20260709 12:20:46.683385 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 54618.6us
W20260709 12:20:46.687280   632 log.cc:927] Time spent T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:20:46.733433   711 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:20:46.768302   696 raft_consensus.cc:993] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:46.768932   696 raft_consensus.cc:1081] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:46.773205 25786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:20:46.773764 25786 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:46.774088 25786 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:46.793262 25786 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:46.813112 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:46.813411 25786 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:46.814031 25706 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:46.814702 25706 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:46.816352 25706 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:46.824373 26338 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56168
I20260709 12:20:46.825007 26338 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:46.825400 26338 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:46.825191 25706 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:46.832408 26338 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:46.877578 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 193902us
I20260709 12:20:46.882336 26338 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:46.883632 25706 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
I20260709 12:20:46.888649 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:46.889442 26338 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:46.893965 25637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:46.894663 25637 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:46.913792   705 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:46.914876   705 tablet_bootstrap.cc:492] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:20:46.915611   705 ts_tablet_manager.cc:1403] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.824s	user 0.050s	sys 0.021s
I20260709 12:20:46.949468 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:46.950129 25630 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:46.952235 25709 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:46.958009 26343 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:46.960604 25630 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
I20260709 12:20:46.965579 26342 raft_consensus.cc:1240] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:46.970965   740 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:46.974375   735 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:46.981775 25751 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:46.982627 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:46.983217 25784 raft_consensus.cc:3055] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:46.983510 25784 raft_consensus.cc:740] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:46.984196 25784 consensus_queue.cc:260] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:46.984804   539 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:46.985324 25784 raft_consensus.cc:3060] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:46.985293   539 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:46.986439   539 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:46.987383   723 raft_consensus.cc:993] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: : Instructing follower 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:20:46.993539   723 raft_consensus.cc:1081] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Signalling peer 0ae979356d694a09be4b0cd976ea354a to start an election
I20260709 12:20:46.995834 25636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:51520
I20260709 12:20:46.996470 25636 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:46.996908 25636 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:46.999361   737 raft_consensus.cc:1064] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:20:47.001431 25780 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:47.003916   735 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:47.009083 26338 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
I20260709 12:20:47.013707 25636 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:47.002153   737 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:47.027719 25784 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:46.993484 25637 raft_consensus.cc:2468] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:47.051321 26307 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:47.061267 25636 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:47.062314 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:47.062983 26338 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:47.072561 25786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:47.089378 26338 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:47.090641 25601 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742; no voters: 
W20260709 12:20:47.125871 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 70260.3us
I20260709 12:20:47.120119   735 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:47.133383 25533 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 21 to 22, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 22 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.160497   671 raft_consensus.cc:2804] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:47.161430   671 raft_consensus.cc:697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:47.209801   705 raft_consensus.cc:359] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:47.211272   705 raft_consensus.cc:740] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:20:47.295970 25786 raft_consensus.cc:3055] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:47.321372 25786 raft_consensus.cc:740] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:47.323242 25786 consensus_queue.cc:260] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:47.324803 25786 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:47.162760   671 consensus_queue.cc:237] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:47.259428   705 consensus_queue.cc:260] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:47.342983   705 ts_tablet_manager.cc:1434] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.427s	user 0.009s	sys 0.003s
W20260709 12:20:47.381299   707 log.cc:927] Time spent T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.107s	user 0.004s	sys 0.000s
W20260709 12:20:47.487099 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 419319us
I20260709 12:20:47.502188 25786 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae979356d694a09be4b0cd976ea354a in term 19.
I20260709 12:20:47.462210 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:47.562927   735 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Becoming Leader. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:47.635135 26341 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:47.635962   707 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:47.662743   735 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:47.707662   707 tablet_bootstrap.cc:492] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
W20260709 12:20:47.740675   724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.013s	sys 0.009s
W20260709 12:20:47.752722   724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.014s	sys 0.009s
I20260709 12:20:47.745462 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-0c2fd18ad19c4394a69e6f9d6aa778e1 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:20:47.774646 26338 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:20:47.777117 26337 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:47.777046 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-0c2fd18ad19c4394a69e6f9d6aa778e1 on tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:20:47.773723   707 ts_tablet_manager.cc:1403] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 1.438s	user 0.067s	sys 0.044s
W20260709 12:20:47.810079   742 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.597s	user 0.005s	sys 0.003s
W20260709 12:20:47.848897 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 78806.4us
W20260709 12:20:47.817039 25751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:47.817955 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 55280.3us
W20260709 12:20:47.854269 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 82288.6us
W20260709 12:20:47.849406   749 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
I20260709 12:20:47.873526 25532 catalog_manager.cc:5697] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 19 to 20, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:47.898751 26341 raft_consensus.cc:1217] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.11->[15.12-15.12]   Dedup: 15.12->[]
I20260709 12:20:47.916754 25704 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:47.927064   707 raft_consensus.cc:359] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:47.928336   707 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:20:47.929076   707 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:47.930713   769 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
W20260709 12:20:47.948944 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 47262.1us
W20260709 12:20:47.972077 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 75559us
I20260709 12:20:47.982404 25630 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:20:47.990089   744 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.385s	user 0.000s	sys 0.002s
I20260709 12:20:47.991827 25637 raft_consensus.cc:1240] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:47.989881   768 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:20:47.981002   711 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.051s	user 0.004s	sys 0.000s
I20260709 12:20:48.011302 25786 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:48.020175 26343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56146 (request call id 1370) took 1041 ms. Trace:
I20260709 12:20:48.020756 26343 rpcz_store.cc:276] 0709 12:20:46.976718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:46.978116 (+  1398us) service_pool.cc:224] Handling call
0709 12:20:46.978445 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:46.986003 (+  7558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:46.986035 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:46.986581 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:20:47.116026 (+129445us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:47.116084 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:47.117330 (+  1246us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:47.284511 (+167181us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b180038ed20. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b28a2522 00007f29b28a47c1 00007f29b28a4779 00007f29b28a4709 00007f29b28a46d1 00007f29b28a39cd 0000000000388634 0000000000388479 00007f29b288583a
0709 12:20:47.989234 (+704723us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b4400022b4c. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b272adb1 00007f29b27c8c85 00007f29b27c96a2 00007f29b553b1db 00007f29b5536aaf 00007f29b552a86e 00007f29b5529bb6 00007f29b55297d8 00007f29b5533221 00007f29b55331d9 00007f29b5533169
0709 12:20:48.017794 (+ 28560us) raft_consensus.cc:1685] finished
0709 12:20:48.017816 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:48.017937 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7954,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"spinlock_wait_cycles":140413184,"thread_start_us":84799,"threads_started":1,"wal-append.queue_time_us":532031}
I20260709 12:20:48.027289 25786 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:20:48.064196 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 50947.3us
I20260709 12:20:48.081024   733 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:20:48.084965   622 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.112s
I20260709 12:20:48.087357   723 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:48.087669   707 ts_tablet_manager.cc:1434] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.240s	user 0.007s	sys 0.002s
I20260709 12:20:48.090943 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-12eaa1f2655a4cc7bde6bd409ccdf780 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:20:48.091536 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-12eaa1f2655a4cc7bde6bd409ccdf780 on tablet 12eaa1f2655a4cc7bde6bd409ccdf780 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:20:48.078442 25528 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: term changed from 18 to 19, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 0ae979356d694a09be4b0cd976ea354a (127.23.217.65). New cstate: current_term: 19 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.096508   740 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
W20260709 12:20:48.126823 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 78434.5us
I20260709 12:20:48.139724 25637 raft_consensus.cc:1275] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:48.155723 26342 raft_consensus.cc:1217] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.8->[16.9-16.9]   Dedup: 16.9->[]
I20260709 12:20:48.205090   711 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:48.206462   711 tablet_bootstrap.cc:492] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:20:48.207397   711 ts_tablet_manager.cc:1403] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 1.474s	user 0.049s	sys 0.020s
W20260709 12:20:48.251562   776 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 12:20:48.289921   783 log.cc:927] Time spent T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.129s	user 0.001s	sys 0.002s
I20260709 12:20:48.289705   711 raft_consensus.cc:359] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:48.290642   711 raft_consensus.cc:740] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:20:48.291384   711 consensus_queue.cc:260] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:20:48.303886 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 146567us
I20260709 12:20:48.331576 26337 raft_consensus.cc:1217] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.10->[15.11-15.11]   Dedup: 15.11->[]
I20260709 12:20:48.350582 26342 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:48.358503   768 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:20:48.373461   711 ts_tablet_manager.cc:1434] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.166s	user 0.007s	sys 0.001s
I20260709 12:20:48.353271   727 consensus_queue.cc:1048] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
W20260709 12:20:48.391685 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 52185.3us
I20260709 12:20:48.419749 25780 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:48.426946 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-7fee45cced0b4ce18dea86254386fee0 received from {username='slave'} at 127.0.0.1:50646
I20260709 12:20:48.440953 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-7fee45cced0b4ce18dea86254386fee0 on tablet 7fee45cced0b4ce18dea86254386fee0 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:20:48.441220   731 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:48.446435   740 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:48.460423   781 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.308s	user 0.004s	sys 0.000s
W20260709 12:20:48.463416 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49626us
W20260709 12:20:48.468295 25522 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8a1f6623a51a47759e59269cfa63be4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica c7bf8f20701245bcbaa09cc663025f97 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:48.484499 25779 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:48.514123 25780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:20:48.514945 25780 raft_consensus.cc:606] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:48.576719 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 97740.9us
I20260709 12:20:48.565677 26337 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:48.588178   794 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 12:20:48.623210 25533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42098 (request call id 313) took 1529 ms. Trace:
I20260709 12:20:48.627206 25532 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56312 (request call id 218) took 1009 ms. Trace:
I20260709 12:20:48.627597 25532 rpcz_store.cc:276] 0709 12:20:47.617540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:47.617812 (+   272us) service_pool.cc:224] Handling call
0709 12:20:48.627114 (+1009302us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:48.232352 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:48.232935 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:48.232964 (+    29us) write_op.cc:435] Acquired schema lock
0709 12:20:48.232983 (+    19us) tablet.cc:662] Decoding operations
0709 12:20:48.233309 (+   326us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:48.233360 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:48.233390 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:48.233578 (+   188us) tablet.cc:701] Row locks acquired
0709 12:20:48.233593 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:20:48.233756 (+   163us) write_op.cc:270] Start()
0709 12:20:48.233863 (+   107us) write_op.cc:276] Timestamp: P: 1783599648233705 usec, L: 0
0709 12:20:48.233879 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:20:48.354166 (+120287us) log.cc:844] Serialized 536 byte log entry
0709 12:20:48.370882 (+ 16716us) op_driver.cc:464] REPLICATION: finished
0709 12:20:48.401103 (+ 30221us) write_op.cc:301] APPLY: starting
0709 12:20:48.401185 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:48.402062 (+   877us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:48.402179 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:48.402869 (+   690us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:48.403004 (+   135us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:48.403044 (+    40us) write_op.cc:312] APPLY: finished
0709 12:20:48.403232 (+   188us) log.cc:844] Serialized 25 byte log entry
0709 12:20:48.403398 (+   166us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:48.403475 (+    77us) write_op.cc:454] Released schema lock
0709 12:20:48.403511 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:48.456117 (+ 52606us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598ae28 00007f29b93c5581 00007f29b93c4d58 00007f29b93c99f1 00007f29b93c99a9 00007f29b93c9939 00007f29b93c9901 00007f29b93c8b2d 0000000000388634 0000000000388479
0709 12:20:48.456351 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:48.456379 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":218984,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":11025,"apply.run_cpu_time_us":2697,"apply.run_wall_time_us":2696,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4186,"num_ops":1,"prepare.queue_time_us":355472,"prepare.run_cpu_time_us":8764,"prepare.run_wall_time_us":223940,"replication_time_us":136880,"spinlock_wait_cycles":165381376}]]}
I20260709 12:20:48.636304 25706 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:48.623713 25533 rpcz_store.cc:276] 0709 12:20:47.094311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:47.094515 (+   204us) service_pool.cc:224] Handling call
0709 12:20:48.623144 (+1528629us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:47.718715 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:47.820953 (+102238us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:47.821008 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:20:47.821031 (+    23us) tablet.cc:662] Decoding operations
0709 12:20:47.821512 (+   481us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:47.821598 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:47.821640 (+    42us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:47.821905 (+   265us) tablet.cc:701] Row locks acquired
0709 12:20:47.821929 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:20:47.822125 (+   196us) write_op.cc:270] Start()
0709 12:20:47.822286 (+   161us) write_op.cc:276] Timestamp: P: 1783599647822058 usec, L: 0
0709 12:20:47.822310 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:20:47.865147 (+ 42837us) log.cc:844] Serialized 599 byte log entry
0709 12:20:48.113497 (+248350us) op_driver.cc:464] REPLICATION: finished
0709 12:20:48.231907 (+118410us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5400001ed4. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29b58f8571 00007f29b598ae28 00007f29b93c5581 00007f29b93c4d58 00007f29b93c99f1 00007f29b93c99a9 00007f29b93c9939 00007f29b93c9901 00007f29b93c8b2d 0000000000388634 0000000000388479
0709 12:20:48.363519 (+131612us) write_op.cc:301] APPLY: starting
0709 12:20:48.363611 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:48.364828 (+  1217us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:48.364852 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:48.365603 (+   751us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:48.365773 (+   170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:48.365824 (+    51us) write_op.cc:312] APPLY: finished
0709 12:20:48.366002 (+   178us) log.cc:844] Serialized 25 byte log entry
0709 12:20:48.366162 (+   160us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:48.366251 (+    89us) write_op.cc:454] Released schema lock
0709 12:20:48.366291 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:48.367063 (+   772us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:48.367095 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":241815,"apply.run_cpu_time_us":3113,"apply.run_wall_time_us":3541,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8313,"num_ops":1,"prepare.queue_time_us":389191,"prepare.run_cpu_time_us":13683,"prepare.run_wall_time_us":573514,"replication_time_us":279726,"spinlock_wait_cycles":30807808,"thread_start_us":391348,"threads_started":2}]]}
I20260709 12:20:48.665925 25786 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "0ae979356d694a09be4b0cd976ea354a"
 from {username='slave'} at 127.0.0.1:36022
W20260709 12:20:48.678145 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 92776.2us
I20260709 12:20:48.687691   765 raft_consensus.cc:1064] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:20:48.696986   765 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:48.715385 25786 raft_consensus.cc:606] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Received request to transfer leadership to TS 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:48.715920 25786 raft_consensus.cc:614] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:48.724807 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:48.726655 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:48.727504 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:48.736090   721 raft_consensus.cc:1064] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:20:48.767748   721 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 7, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:48.777563   792 raft_consensus.cc:1064] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:20:48.762954 25532 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:48.787642   792 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:48.813514 26339 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:48.816084 25784 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:48.818228   659 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
W20260709 12:20:48.916877   797 log.cc:927] Time spent T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:20:48.922207 26339 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:48.928736   726 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:48.946895   762 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:48.972750 25784 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:48.979409 25636 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:48.997560   610 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:49.002862   667 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
W20260709 12:20:49.126269   795 log.cc:927] Time spent T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:20:49.165611 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 184228us
W20260709 12:20:49.171624 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 188271us
W20260709 12:20:49.170478 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 174278us
I20260709 12:20:49.231863 26339 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:49.257377 25782 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:20:49.283133 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 280441us
I20260709 12:20:49.295673 25630 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:49.277012   721 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:20:49.369225   610 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:20:49.396987 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 209506us
W20260709 12:20:49.409857 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 99027.5us
W20260709 12:20:49.412567 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 105824us
W20260709 12:20:49.450887 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 138016us
W20260709 12:20:49.467504 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 161663us
W20260709 12:20:49.518826   816 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:20:49.574637   722 raft_consensus.cc:993] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:20:49.576726   813 raft_consensus.cc:1081] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
W20260709 12:20:49.622241 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 332331us
I20260709 12:20:49.602887   761 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:49.636858 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 164785us
I20260709 12:20:49.643774   832 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:49.670986   761 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:49.728709 25630 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:49.754200 25707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57966
I20260709 12:20:49.754976 25707 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:49.755365 25707 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:49.778899   761 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:49.782936   811 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:49.783344 25707 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:49.785156 25707 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Requested vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:49.796154 25633 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:49.801496 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 146839us
W20260709 12:20:49.802618 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 336113us
W20260709 12:20:49.809850 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 77265.1us
I20260709 12:20:49.873554 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:49.874207 25782 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:49.874521 25782 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:49.877187 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 74922.4us
I20260709 12:20:49.877557 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:49.880609 26342 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:49.949877 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 147560us
I20260709 12:20:49.949644 25782 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:49.951251 25782 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:49.891503 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:50.008446 25782 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:50.009650 26342 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
W20260709 12:20:50.026345 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 140999us
W20260709 12:20:50.032996 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 162100us
I20260709 12:20:50.043432 25708 raft_consensus.cc:1240] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Rejecting Update request from peer 955233c139344131a847044f17f8d348 for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:50.044550 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 165925us
W20260709 12:20:50.086010   838 log.cc:927] Time spent T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:20:50.097922   833 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.146240   734 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:50.147142   734 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:50.175714   734 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:50.189595   814 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:50.195163 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:50.197105   814 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:50.209086   814 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:50.222360 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 176910us
W20260709 12:20:50.229619 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 192430us
I20260709 12:20:50.232184 25676 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:50.288957 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 56866.8us
I20260709 12:20:50.290050   610 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:50.290674   610 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:50.290766 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:50.291630   610 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:50.294472 26309 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:50.296593   662 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:50.329270 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:50.331305 25783 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:50.339154   852 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:20:50.360740 26337 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.364038   726 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:20:50.372952 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 490350us
W20260709 12:20:50.386008 25602 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:50.402875 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:50.419164 26342 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.397491 25633 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.440182 25784 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.433679 25677 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:20:50.445600 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 69604.3us
I20260709 12:20:50.442286 26339 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.459621 25630 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.459684 25532 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 19 to 20, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 20 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.468092   823 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:50.496846 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 546045us
W20260709 12:20:50.512643   855 log.cc:927] Time spent T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:20:50.519800   853 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.523226 26342 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:50.530871 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:50.533403 25787 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:50.543802 25531 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 reported cstate change: config changed from index 12 to 13, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.588218   528 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:50.589314 25629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:50.613415 25533 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 9 to 10, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.617081 25633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13"
dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
mode: GRACEFUL
new_leader_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:60940
I20260709 12:20:50.615790 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:50.619820 25633 raft_consensus.cc:606] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Received request to transfer leadership to TS c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:50.661572 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 47282.2us
W20260709 12:20:50.673034 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 65543.6us
W20260709 12:20:50.685218 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 58451.6us
I20260709 12:20:50.621876   761 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:20:50.717530 25787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b"
dest_uuid: "955233c139344131a847044f17f8d348"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:36022
I20260709 12:20:50.718163 25787 raft_consensus.cc:606] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:20:50.729423 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:50.731616 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:50.780706 26342 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:20:50.798266 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 175739us
W20260709 12:20:50.804093 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 120740us
W20260709 12:20:50.805006 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 131173us
I20260709 12:20:50.787015   823 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:20:50.811275 25603 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:50.838423   737 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:50.867977 25779 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:50.900385   750 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:50.928735   761 raft_consensus.cc:993] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: : Instructing follower c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:20:50.929307   761 raft_consensus.cc:1081] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Signalling peer c7bf8f20701245bcbaa09cc663025f97 to start an election
I20260709 12:20:50.932546 25708 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
 from {username='slave'} at 127.0.0.1:57968
I20260709 12:20:50.933117 25708 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:50.933457 25708 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:50.933776 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:50.965920   822 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:50.968758 25708 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:50.986191 25708 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:51.007597 25784 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.013782 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:51.014390 25780 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:51.018253 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:51.018831 25636 raft_consensus.cc:3055] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:51.019091 25636 raft_consensus.cc:740] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ae979356d694a09be4b0cd976ea354a, State: Running, Role: LEADER
I20260709 12:20:51.020008 25636 consensus_queue.cc:260] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:51.021260 25636 raft_consensus.cc:3060] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:51.025198 25780 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:51.039609   812 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:51.047689 25784 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.060456   812 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:51.062915   812 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:51.076897 25638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:51.076421 25633 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.097960 25676 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:51.098867   877 raft_consensus.cc:2804] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:20:51.104370   877 raft_consensus.cc:697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:51.105268   877 consensus_queue.cc:237] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:51.107149 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:51.133435   771 raft_consensus.cc:993] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:51.133950   771 raft_consensus.cc:1081] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:51.138149 25636 raft_consensus.cc:2468] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:51.144105 26341 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56168
I20260709 12:20:51.152561   822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:56416 (request call id 15) took 1360 ms. Trace:
I20260709 12:20:51.160915 26341 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.161520 26341 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:51.157262   822 rpcz_store.cc:276] 0709 12:20:49.792370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.793878 (+  1508us) service_pool.cc:224] Handling call
0709 12:20:49.799943 (+  6065us) log.cc:844] Serialized 501 byte log entry
0709 12:20:50.371480 (+571537us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b180049d500. stack: 00007f29b2870343 00007f29b2870240 00007f29b21ff1d4 00007f29bbf3d304 00007f29bbf3d299 00007f29bb76aab1 00007f29bb777281 00007f29b281cc0a 00007f29bbf5c219 00007f29b28a4a40 00007f29b289f4e4 00007f29b289f455 00007f29b58e8b0b 00007f29b596db63 00007f29b5995752 00007f29b598d9fa
0709 12:20:51.152458 (+780978us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":374018,"raft.queue_time_us":521431,"raft.run_cpu_time_us":1575,"raft.run_wall_time_us":126175,"spinlock_wait_cycles":8154368,"thread_start_us":58102,"threads_started":1,"wal-append.queue_time_us":175742}
I20260709 12:20:51.166002 25521 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2c9b915f0d524c19b87995309e7ea872 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:51.170183 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:51.195705   877 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:51.226054 26335 raft_consensus.cc:2955] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.238461 26341 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:51.260325   877 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:51.261106   877 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:51.277447 25531 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 12 to 13, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 16 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.278837 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0ae979356d694a09be4b0cd976ea354a" is_pre_election: true
I20260709 12:20:51.279508 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:51.280160 25708 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:51.282023   877 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:51.287639 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:51.288332 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:51.289330 25676 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:51.290225 26341 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:51.288465 25708 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 23.
I20260709 12:20:51.291776   877 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:51.302953 25531 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 11 to 12, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.319783 25636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:51.320483 25636 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:51.331022 25636 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 23.
I20260709 12:20:51.342307 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:51.342984 25784 raft_consensus.cc:3055] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:51.343284 25784 raft_consensus.cc:740] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:51.344056 25784 consensus_queue.cc:260] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:51.345525 25784 raft_consensus.cc:3060] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:51.356645 25784 raft_consensus.cc:2468] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 23.
I20260709 12:20:51.359297 26308 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:51.360289   837 raft_consensus.cc:2804] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:51.360884   837 raft_consensus.cc:697] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 23 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:51.361940   837 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:51.402848 25532 catalog_manager.cc:5697] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 11 to 12, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 12 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.429560 25708 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:51.436622 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c9b915f0d524c19b87995309e7ea872 with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:51.439436 26342 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:51.441339   877 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:51.450353   833 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.452199 26341 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.503273 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12eaa1f2655a4cc7bde6bd409ccdf780 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:51.504007 25636 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:51.523849   877 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:51.520012 25707 consensus_queue.cc:237] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:51.530207 25712 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:51.546169   885 tablet_replica.cc:333] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:20:51.547385   885 raft_consensus.cc:2243] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:51.548242   885 raft_consensus.cc:2272] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:51.550034 25779 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:51.551461 26342 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:51.558686   823 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:20:51.567823   833 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:51.580338 25636 raft_consensus.cc:2955] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.594278 25784 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.585376   885 ts_tablet_manager.cc:1916] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:51.606868 25633 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 12:20:51.607926   884 log.cc:927] Time spent T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:20:51.608417   831 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:51.632531 26342 raft_consensus.cc:1275] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:20:51.637158   750 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.642401 25532 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 22 to 23, leader changed from 955233c139344131a847044f17f8d348 (127.23.217.67) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 23 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:51.646853 25533 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a reported cstate change: config changed from index 10 to 11, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 16 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.667433   726 consensus_queue.cc:1048] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:51.707253 25532 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 12 to 13, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.721033 26341 raft_consensus.cc:2955] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.743686   812 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Committing config change with OpId 15.14: config changed from index 13 to 14, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.747799 25619 tablet_service.cc:1558] Processing DeleteTablet for tablet 7fee45cced0b4ce18dea86254386fee0 with delete_type TABLET_DATA_TOMBSTONED (TS 0ae979356d694a09be4b0cd976ea354a not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:56416
I20260709 12:20:51.750820 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 12eaa1f2655a4cc7bde6bd409ccdf780 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:40804
I20260709 12:20:51.765367 26335 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Committing config change with OpId 15.14: config changed from index 13 to 14, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.828671 25531 catalog_manager.cc:5697] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 13 to 14, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New cstate: current_term: 15 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:51.833590 25784 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:20:51.852870 25770 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 with delete_type TABLET_DATA_TOMBSTONED (TS 955233c139344131a847044f17f8d348 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:40804
I20260709 12:20:51.863914   830 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:51.883351 25708 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:51.885993   865 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:51.898936   888 log.cc:927] Time spent T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: Append to log took a long time: real 0.176s	user 0.004s	sys 0.001s
I20260709 12:20:51.911841   891 tablet_replica.cc:333] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:20:51.913473   891 raft_consensus.cc:2243] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:51.914193   890 tablet_replica.cc:333] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:51.915392   890 raft_consensus.cc:2243] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:51.928464   891 raft_consensus.cc:2272] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:51.936326   891 ts_tablet_manager.cc:1916] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:51.938316   890 raft_consensus.cc:2272] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:51.952435   890 ts_tablet_manager.cc:1916] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:20:51.971792 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 186760us
I20260709 12:20:51.983978 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:51.984807 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7fee45cced0b4ce18dea86254386fee0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:51.986634 25633 raft_consensus.cc:1275] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:20:51.978191 25634 raft_consensus.cc:2955] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Committing config change with OpId 15.14: config changed from index 13 to 14, VOTER 955233c139344131a847044f17f8d348 (127.23.217.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:51.999179 25779 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:51.999500 25633 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:52.019729   885 ts_tablet_manager.cc:1929] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.12
I20260709 12:20:52.020421   885 log.cc:1199] T 2c9b915f0d524c19b87995309e7ea872 P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/2c9b915f0d524c19b87995309e7ea872
I20260709 12:20:52.012997   726 consensus_queue.cc:1048] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:20:52.011255   821 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:20:52.058313   826 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:20:52.086755 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76701.6us
I20260709 12:20:52.094321 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet 2c9b915f0d524c19b87995309e7ea872 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:52.190145   890 ts_tablet_manager.cc:1929] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.12
I20260709 12:20:52.190694   890 log.cc:1199] T 7fee45cced0b4ce18dea86254386fee0 P 0ae979356d694a09be4b0cd976ea354a: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/wals/7fee45cced0b4ce18dea86254386fee0
I20260709 12:20:52.199111 25521 catalog_manager.cc:5028] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): tablet 7fee45cced0b4ce18dea86254386fee0 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:52.306042   891 ts_tablet_manager.cc:1929] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.10
I20260709 12:20:52.314774   891 log.cc:1199] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/12eaa1f2655a4cc7bde6bd409ccdf780
I20260709 12:20:52.332571 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 12eaa1f2655a4cc7bde6bd409ccdf780 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:52.332571   891 tablet_replica.cc:333] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:20:52.337659   891 raft_consensus.cc:2243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:52.340281   891 raft_consensus.cc:2272] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:52.349861   891 ts_tablet_manager.cc:1916] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:52.371169   891 ts_tablet_manager.cc:1929] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.13
I20260709 12:20:52.371685   891 log.cc:1199] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 955233c139344131a847044f17f8d348: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/wals/0c2fd18ad19c4394a69e6f9d6aa778e1
I20260709 12:20:52.373338 25520 catalog_manager.cc:5028] TS 955233c139344131a847044f17f8d348 (127.23.217.67:41061): tablet 0c2fd18ad19c4394a69e6f9d6aa778e1 (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:52.537356 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:52.593632 26335 consensus_queue.cc:237] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:52.604027 25779 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:52.605832   864 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:52.616523 25633 raft_consensus.cc:1275] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:52.617412   826 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 3 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:52.619379 25787 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 3 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:52.625037   864 consensus_queue.cc:1048] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:20:52.651937 25533 catalog_manager.cc:5697] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 3 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New cstate: current_term: 23 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:52.653151 25633 raft_consensus.cc:2955] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 3 to 16, VOTER c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:52.925539 25520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8a1f6623a51a47759e59269cfa63be4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
W20260709 12:20:52.936347 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 199750us
I20260709 12:20:53.248037 25694 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a1f6623a51a47759e59269cfa63be4b with delete_type TABLET_DATA_TOMBSTONED (TS c7bf8f20701245bcbaa09cc663025f97 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:35600
I20260709 12:20:53.281985 25711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:52030
I20260709 12:20:53.284611 25711 raft_consensus.cc:606] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Received request to transfer leadership to TS 10d0c7349085463595aedaf7c071e742
I20260709 12:20:53.299414   812 raft_consensus.cc:993] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:53.302022   726 raft_consensus.cc:1081] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Signalling peer 10d0c7349085463595aedaf7c071e742 to start an election
I20260709 12:20:53.314057   920 tablet_replica.cc:333] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:20:53.317709   920 raft_consensus.cc:2243] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.324146   920 raft_consensus.cc:2272] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.336463 26338 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780"
dest_uuid: "10d0c7349085463595aedaf7c071e742"
 from {username='slave'} at 127.0.0.1:56156
I20260709 12:20:53.337224 26338 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:53.337633 26338 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:53.338438   920 ts_tablet_manager.cc:1916] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:53.338928 25705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
new_leader_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:52030
I20260709 12:20:53.351644 25705 raft_consensus.cc:606] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Received request to transfer leadership to TS 955233c139344131a847044f17f8d348
I20260709 12:20:53.357177 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:53.360114 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:53.362306 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:53.361465 26338 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:53.368096 25630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "0ae979356d694a09be4b0cd976ea354a"
I20260709 12:20:53.369035 25630 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:53.375133 26338 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:53.375458 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:53.376230 25705 raft_consensus.cc:3055] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:53.376571 25705 raft_consensus.cc:740] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:53.377676 25705 consensus_queue.cc:260] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:53.379352 25705 raft_consensus.cc:3060] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:53.405658 25705 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:53.410446 26309 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:53.408663 25630 raft_consensus.cc:2468] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
I20260709 12:20:53.411566   826 raft_consensus.cc:2804] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:53.412101   826 raft_consensus.cc:697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:53.413223   826 consensus_queue.cc:237] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:53.453038 25533 catalog_manager.cc:5697] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 16 to 17, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 17 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:53.468132   921 raft_consensus.cc:993] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:53.468920   921 raft_consensus.cc:1081] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:20:53.473084 25783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:20:53.473610 25783 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:53.473997 25783 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:53.484412 25783 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:53.494282   920 ts_tablet_manager.cc:1929] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.15
I20260709 12:20:53.503648   920 log.cc:1199] T 8a1f6623a51a47759e59269cfa63be4b P c7bf8f20701245bcbaa09cc663025f97: Deleting WAL directory at /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/wals/8a1f6623a51a47759e59269cfa63be4b
I20260709 12:20:53.506848 25522 catalog_manager.cc:5028] TS c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): tablet 8a1f6623a51a47759e59269cfa63be4b (table test-workload [id=7b895a80fe4046338b656f39aee06a3c]) successfully deleted
I20260709 12:20:53.530709 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:53.532138 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593fa615e9b64ba0974d4a40b5307c3c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:53.532747 26342 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:53.531482 25711 raft_consensus.cc:3055] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:53.534950 25711 raft_consensus.cc:740] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:53.541162 26342 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:53.530732 25783 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:53.543352 25711 consensus_queue.cc:260] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:53.544844 25750 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:53.554955 25711 raft_consensus.cc:3060] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:53.555649   810 raft_consensus.cc:2804] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:53.556289   810 raft_consensus.cc:697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 21 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:53.557291   810 consensus_queue.cc:237] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:53.572616 25711 raft_consensus.cc:2468] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:53.631855 25533 catalog_manager.cc:5697] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 20 to 21, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 21 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:53.688058 25630 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:20:53.724475 25601 consensus_peers.cc:597] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bfd362b7abdb43de96e8b4d518fbb0f9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.727186 25783 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:20:53.728863 25711 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:20:53.730422   916 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:53.733096   914 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:53.749557   914 raft_consensus.cc:2955] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index -1 to 16, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:53.752939 25707 raft_consensus.cc:2955] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index -1 to 16, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:53.775422 25533 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index -1 to 16, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 21 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:53.800092 25783 raft_consensus.cc:2955] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index -1 to 16, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:53.839254 25711 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:20:53.865703 25676 consensus_peers.cc:597] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e905093d8adb4c6797cd77172a30843c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.870077 25630 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:20:53.874940   922 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:53.878130 25779 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:20:53.888206   922 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:53.893899 25634 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:53.902057   826 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:53.931047 25707 raft_consensus.cc:1275] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:53.934576   910 consensus_queue.cc:1048] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:53.947351   921 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:53.954260 25784 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:54.003311 25633 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:54.104405 25711 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:54.106087   908 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:20:54.112587 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 85077.4us
W20260709 12:20:54.123744 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 57516.2us
I20260709 12:20:54.178948 26342 raft_consensus.cc:1275] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:54.200131 25531 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 10, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 14 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:54.202703   908 consensus_queue.cc:1048] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:54.211256 25636 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:54.263679   942 ts_tablet_manager.cc:933] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
I20260709 12:20:54.332909 25784 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:54.334558   822 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:54.337265 25601 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a -> Peer 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): Couldn't send request to peer 10d0c7349085463595aedaf7c071e742. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 294a99dd73c8449f84538cf8965a6467. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:54.348317 25708 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Refusing update from remote peer 0ae979356d694a09be4b0cd976ea354a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:54.380384   916 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:54.387959   942 tablet_copy_client.cc:323] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:20:54.390985 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet bfd362b7abdb43de96e8b4d518fbb0f9 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:20:54.391554 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet bfd362b7abdb43de96e8b4d518fbb0f9 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-bfd362b7abdb43de96e8b4d518fbb0f9
I20260709 12:20:54.424108 25648 tablet_copy_source_session.cc:215] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:54.429076 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:54.430747 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 184959us
I20260709 12:20:54.447095   822 raft_consensus.cc:2955] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:54.453388 25779 raft_consensus.cc:2955] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:54.482051 25708 raft_consensus.cc:2955] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:54.483283   942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd362b7abdb43de96e8b4d518fbb0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:54.490854 25528 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index -1 to 13, NON_VOTER 10d0c7349085463595aedaf7c071e742 (127.23.217.68) added. New cstate: current_term: 21 leader_uuid: "0ae979356d694a09be4b0cd976ea354a" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:20:54.547562   954 ts_tablet_manager.cc:933] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:54.575037   954 tablet_copy_client.cc:323] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.66:39947
I20260709 12:20:54.581164   942 tablet_copy_client.cc:806] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:54.581888   942 tablet_copy_client.cc:670] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:54.586613   942 tablet_copy_client.cc:538] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:54.599977 25723 tablet_copy_service.cc:140] P c7bf8f20701245bcbaa09cc663025f97: Received BeginTabletCopySession request for tablet e905093d8adb4c6797cd77172a30843c from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:50646)
I20260709 12:20:54.600646 25723 tablet_copy_service.cc:161] P c7bf8f20701245bcbaa09cc663025f97: Beginning new tablet copy session on tablet e905093d8adb4c6797cd77172a30843c from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:50646: session id = 10d0c7349085463595aedaf7c071e742-e905093d8adb4c6797cd77172a30843c
I20260709 12:20:54.621404 25723 tablet_copy_source_session.cc:215] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:54.635182   954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e905093d8adb4c6797cd77172a30843c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:54.647567   942 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:20:54.677620   954 tablet_copy_client.cc:806] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:54.690928   954 tablet_copy_client.cc:670] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:54.698191   954 tablet_copy_client.cc:538] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:54.832753   954 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
I20260709 12:20:54.834720 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:54.848624 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:54.849392 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:54.852140   957 ts_tablet_manager.cc:933] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: Initiating tablet copy from peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483)
W20260709 12:20:54.917263 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 97256.5us
W20260709 12:20:54.996977 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 107957us
I20260709 12:20:55.048048   957 tablet_copy_client.cc:323] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: tablet copy: Beginning tablet copy session from remote peer at address 127.23.217.65:41483
I20260709 12:20:55.071343 25648 tablet_copy_service.cc:140] P 0ae979356d694a09be4b0cd976ea354a: Received BeginTabletCopySession request for tablet 294a99dd73c8449f84538cf8965a6467 from peer 10d0c7349085463595aedaf7c071e742 ({username='slave'} at 127.0.0.1:60736)
I20260709 12:20:55.072127 25648 tablet_copy_service.cc:161] P 0ae979356d694a09be4b0cd976ea354a: Beginning new tablet copy session on tablet 294a99dd73c8449f84538cf8965a6467 from peer 10d0c7349085463595aedaf7c071e742 at {username='slave'} at 127.0.0.1:60736: session id = 10d0c7349085463595aedaf7c071e742-294a99dd73c8449f84538cf8965a6467
W20260709 12:20:55.083868 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 53515us
I20260709 12:20:55.175377 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:55.206668 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 60898.4us
I20260709 12:20:55.209605 25648 tablet_copy_source_session.cc:215] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:55.213673   957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 294a99dd73c8449f84538cf8965a6467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:55.266737   954 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.268978   954 tablet_bootstrap.cc:492] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:20:55.269867   954 ts_tablet_manager.cc:1403] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.437s	user 0.041s	sys 0.009s
I20260709 12:20:55.262277   942 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.273106   942 tablet_bootstrap.cc:492] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:20:55.273901   942 ts_tablet_manager.cc:1403] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.627s	user 0.055s	sys 0.031s
I20260709 12:20:55.303682   954 raft_consensus.cc:359] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:55.304591   954 raft_consensus.cc:740] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:20:55.304877   942 raft_consensus.cc:359] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:55.305266   954 consensus_queue.cc:260] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:55.305893   942 raft_consensus.cc:740] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
I20260709 12:20:55.306766   942 consensus_queue.cc:260] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:55.318425   957 tablet_copy_client.cc:806] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:55.319092   957 tablet_copy_client.cc:670] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:55.321600   954 ts_tablet_manager.cc:1434] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 12:20:55.330616   957 tablet_copy_client.cc:538] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:55.333361   942 ts_tablet_manager.cc:1434] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 12:20:55.354943 25648 tablet_copy_service.cc:342] P 0ae979356d694a09be4b0cd976ea354a: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-bfd362b7abdb43de96e8b4d518fbb0f9 received from {username='slave'} at 127.0.0.1:60736
I20260709 12:20:55.355417 25648 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-bfd362b7abdb43de96e8b4d518fbb0f9 on tablet bfd362b7abdb43de96e8b4d518fbb0f9 with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:20:55.372079 25723 tablet_copy_service.cc:342] P c7bf8f20701245bcbaa09cc663025f97: Request end of tablet copy session 10d0c7349085463595aedaf7c071e742-e905093d8adb4c6797cd77172a30843c received from {username='slave'} at 127.0.0.1:50646
I20260709 12:20:55.372520 25723 tablet_copy_service.cc:434] P c7bf8f20701245bcbaa09cc663025f97: ending tablet copy session 10d0c7349085463595aedaf7c071e742-e905093d8adb4c6797cd77172a30843c on tablet e905093d8adb4c6797cd77172a30843c with peer 10d0c7349085463595aedaf7c071e742
I20260709 12:20:55.378824 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
W20260709 12:20:55.386056 26308 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:55.431586   957 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: Bootstrap starting.
W20260709 12:20:55.462692 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59230.5us
W20260709 12:20:55.459254 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 73379.5us
I20260709 12:20:55.513813 26342 raft_consensus.cc:1217] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.9->[14.10-14.10]   Dedup: 14.10->[]
W20260709 12:20:55.522217 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.542869 25677 consensus_peers.cc:597] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.545341 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.552330 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.550597 25677 consensus_peers.cc:597] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.554795 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.566344 25677 consensus_peers.cc:597] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.563823 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.569252 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.577582 24421 tablet_copy_service.cc:362] P 0ae979356d694a09be4b0cd976ea354a: Destroying tablet copy session 10d0c7349085463595aedaf7c071e742-294a99dd73c8449f84538cf8965a6467 due to service shutdown
I20260709 12:20:55.578060 24421 tablet_copy_service.cc:434] P 0ae979356d694a09be4b0cd976ea354a: ending tablet copy session 10d0c7349085463595aedaf7c071e742-294a99dd73c8449f84538cf8965a6467 on tablet 294a99dd73c8449f84538cf8965a6467 with peer 10d0c7349085463595aedaf7c071e742
W20260709 12:20:55.586475 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.595914 25677 consensus_peers.cc:597] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.748253 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.748893 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 145460us
I20260709 12:20:55.755657   957 tablet_bootstrap.cc:700] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: Time spent opening tablet: real 0.120s	user 0.000s	sys 0.016s
W20260709 12:20:55.761906 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 159206us
W20260709 12:20:55.985395 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 80442.9us
I20260709 12:20:56.019436   957 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.020857   957 tablet_bootstrap.cc:492] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: Bootstrap complete.
I20260709 12:20:56.021925   957 ts_tablet_manager.cc:1403] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: Time spent bootstrapping tablet: real 0.591s	user 0.045s	sys 0.041s
W20260709 12:20:56.025691 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 52802.2us
W20260709 12:20:56.044018 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 71051.3us
W20260709 12:20:56.053597 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 75562.5us
I20260709 12:20:56.060791 26342 raft_consensus.cc:1217] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.15->[21.16-21.16]   Dedup: 21.16->[]
W20260709 12:20:56.065603 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 221305us
W20260709 12:20:56.278097 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 203843us
I20260709 12:20:56.287164   957 raft_consensus.cc:359] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:56.288549   957 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Initialized, Role: LEARNER
W20260709 12:20:56.311259 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 253793us
I20260709 12:20:56.316138   957 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:20:56.341651 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 286405us
I20260709 12:20:56.359501   957 ts_tablet_manager.cc:1434] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: Time spent starting tablet: real 0.337s	user 0.011s	sys 0.008s
I20260709 12:20:56.523973   974 raft_consensus.cc:493] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.527475   916 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:56.527631   974 raft_consensus.cc:515] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.528133   916 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.528744 26343 raft_consensus.cc:1217] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.12->[21.13-21.13]   Dedup: 21.13->[]
I20260709 12:20:56.530037   974 leader_election.cc:290] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.530131   916 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.543951 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.550702 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.543437   916 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.541356   974 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:56.579537 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 238908us
W20260709 12:20:56.585347 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.586828 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a1f6623a51a47759e59269cfa63be4b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.597841 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13025a828f141378027526ae4f0b002" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.599647   990 raft_consensus.cc:1064] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:20:56.603185   973 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:56.603849   973 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.605928   973 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.585239   914 raft_consensus.cc:493] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.607348   914 raft_consensus.cc:515] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.609297   914 leader_election.cc:290] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.587826   916 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:56.613070   916 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:56.612651   990 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.591073   974 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:56.617676   974 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:56.629796 25601 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:56.631081   983 raft_consensus.cc:493] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.631857   983 raft_consensus.cc:515] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.641927 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:56.643288 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 51816.1us
I20260709 12:20:56.643889 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1ee3e27f67449aaab6c88508697a762" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.644122 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.639861   822 raft_consensus.cc:493] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.645824   822 raft_consensus.cc:515] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.648000   822 leader_election.cc:290] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.634263   987 raft_consensus.cc:493] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.648705   987 raft_consensus.cc:515] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.634905   980 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.649919   980 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.634402   983 leader_election.cc:290] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.650974   987 leader_election.cc:290] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.635150   994 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:56.651957   980 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.635448   993 raft_consensus.cc:493] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.652904   993 raft_consensus.cc:515] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.655097   993 leader_election.cc:290] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.656812 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.656852 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.661262 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a741c75d1e684375b51f415fd0aa0296" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.665340 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.674022 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36db04d76d874d328966edd6be42e2c8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:56.674604   962 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.675504   962 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:56.688791 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4174bd2da7548489ad8aa892dabb8bb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:56.692546 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa65389a0ed4ac1b3ec08bd217d5f22" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:56.693874   962 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:56.687510   916 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.695570   916 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:56.703245   916 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:56.707474 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:56.709543 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:56.710873 25751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 121546us
I20260709 12:20:56.714182   972 raft_consensus.cc:493] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.714840   972 raft_consensus.cc:515] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.715881 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.716993   972 leader_election.cc:290] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.717451 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.717981 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.721194 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:56.710479 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 82781us
I20260709 12:20:56.726094 25601 leader_election.cc:304] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:56.724815   916 raft_consensus.cc:493] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.751854 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c30f8fbe9ff44588b4847921bab9bd0" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.750115 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9052f9b88ca4b9ebb24b8764ad345af" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:56.759625 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 87194.6us
I20260709 12:20:56.760972 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2fd18ad19c4394a69e6f9d6aa778e1" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:56.755596 25677 consensus_peers.cc:597] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.762148 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f7c369dcd245eeba8804ccb208c79a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:56.743852   914 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.764417 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0024aee53c2a47e59a06e7e69bfb9eb2" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:56.764710 25601 leader_election.cc:304] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:56.765007 25677 consensus_peers.cc:597] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.770925 25601 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:56.771148 25677 consensus_peers.cc:597] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.784792   972 raft_consensus.cc:2749] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:56.785511   997 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:56.806845   914 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.923379   980 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.924088   980 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.925411   914 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.808385 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12eaa1f2655a4cc7bde6bd409ccdf780" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.753341   916 raft_consensus.cc:515] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.769928   987 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.928668   914 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.929306   914 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:56.919757   973 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.920580   975 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:56.930768   916 leader_election.cc:290] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.931185   975 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.933493   975 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.933610   980 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.914685 25603 leader_election.cc:304] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:56.921896   974 raft_consensus.cc:493] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.930801   973 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:56.942521   974 raft_consensus.cc:515] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.943846   973 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:56.944819   974 leader_election.cc:290] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:56.922343   962 raft_consensus.cc:493] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:56.930063   997 raft_consensus.cc:493] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:20:56.945691   962 raft_consensus.cc:515] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:56.945788   997 raft_consensus.cc:515] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:56.921224   993 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:56.948084   962 leader_election.cc:290] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:56.950538 25676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:56.753679   983 raft_consensus.cc:2749] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:56.935114 25601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 149789us
W20260709 12:20:56.956004 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 179443us
W20260709 12:20:56.956537 25603 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 184512us
I20260709 12:20:56.957298   962 raft_consensus.cc:2749] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:56.955065   993 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:56.959928   993 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:56.955914   987 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:56.956780 25677 consensus_peers.cc:597] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.948117   997 leader_election.cc:290] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:56.966693 25677 consensus_peers.cc:597] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.971386 26339 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:56.974303 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.973663 26341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.977524 25783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb52f5946670453a8ec9b21d12ececbb" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.977768 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.979748 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb89b7cd7f4d4662ab9d1e1f55af650c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:56.982858 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.985353 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:56.981735 25677 consensus_peers.cc:597] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.989408 25603 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:56.993383 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:56.994621 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:56.995448   914 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:56.995659 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:56.996789 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.000221 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:57.001703 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.002351 25677 consensus_peers.cc:597] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.010250 25603 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.011164 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.011406 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:57.013235   983 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:57.013190 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a039e6df55747e1b662ecb5d069264b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:56.988334   997 raft_consensus.cc:493] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
W20260709 12:20:57.031845 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 59979.6us
W20260709 12:20:57.033088 25677 consensus_peers.cc:597] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.030628 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.039057 25601 leader_election.cc:304] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:57.008190   994 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:20:57.036024 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 53035.7us
W20260709 12:20:57.032577 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53435.1us
W20260709 12:20:57.037676 25677 consensus_peers.cc:597] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.043215 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.032337   973 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:57.049161 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f80a8201dcaf4f2a9290c47595f9cf7b" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.049928 25601 leader_election.cc:304] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:57.040501  1004 raft_consensus.cc:493] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:57.052763 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4fd24cfa184aa8a8a1de8f32296e6a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:57.053112   980 raft_consensus.cc:2749] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:57.053977   973 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:57.066938   973 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:57.036887   997 raft_consensus.cc:515] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:57.055428 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:57.042584   914 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:57.058925 25601 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.069240   914 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:57.069609 25601 leader_election.cc:304] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.070555   914 raft_consensus.cc:2749] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:57.073583 24421 tablet_replica.cc:333] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.074859 24421 raft_consensus.cc:2243] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.048136   994 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:57.075471 24421 raft_consensus.cc:2272] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.075976  1004 raft_consensus.cc:515] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:57.060817   975 raft_consensus.cc:2749] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:57.092384  1004 leader_election.cc:290] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:57.064559 25780 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:57.067200  1006 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:57.054272 25603 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.094476 25603 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.097751   975 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:57.098371   975 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:57.099213 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5735f590eff4fc9ae2686d09f8b6810" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.104422 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6873460634e2eb3253677a6a6b857" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.104969 25705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.105410 25713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.107647 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.109364   987 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:57.060084 25677 consensus_peers.cc:597] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.080765   973 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:20:57.109700 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.110114   973 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:57.110584 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6158b789bb894b2b9ad0c019dde5476f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.112821   973 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:57.113461 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 53429.3us
I20260709 12:20:57.085738   997 leader_election.cc:290] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:57.087049 24421 tablet_replica.cc:333] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
W20260709 12:20:57.088094 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.116163 24421 raft_consensus.cc:2243] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.117131 24421 raft_consensus.cc:2272] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.088368   994 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:57.124776   997 raft_consensus.cc:493] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:20:57.125324   997 raft_consensus.cc:515] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:57.127424   997 leader_election.cc:290] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:57.128446 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:57.131330 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb70fbead204d9581ca707c05f2befe" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.132109 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.134151   996 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:57.136634 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.138374 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a18426c9efd45e698b2b2ccb80a49e8" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.140972 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.146862 25603 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.141153 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.152417 24421 tablet_replica.cc:333] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.153502 24421 raft_consensus.cc:2243] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.157011 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.159093 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:57.140972 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d711fa725d244128d79dfdfb1e70af6" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.164999 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a36d5c7d94b8ca6fed2b3e1b9ae13" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.165627 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85b6713c7d844fb7a5272cac45763c0a" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.166949 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3c8e03aa6454e118cd3f9f25c785bef" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:57.173336  1004 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:57.180807 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571c5e0767064f3e86479b794294d327" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:57.192060 24421 raft_consensus.cc:2272] T 36db04d76d874d328966edd6be42e2c8 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.193789 25603 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.194860   997 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:57.199633 24421 tablet_replica.cc:333] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.200523 24421 raft_consensus.cc:2243] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.201077 24421 raft_consensus.cc:2272] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.205360 25601 leader_election.cc:304] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:57.216331  1004 raft_consensus.cc:2749] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:57.227856 25601 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.230185 25601 leader_election.cc:304] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:57.235059   997 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:57.235776  1004 raft_consensus.cc:2749] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:57.237097 25601 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
W20260709 12:20:57.237754 25601 raft_consensus.cc:2686] T ef4fd24cfa184aa8a8a1de8f32296e6a P 0ae979356d694a09be4b0cd976ea354a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:57.238570 25601 leader_election.cc:304] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:57.252580   997 raft_consensus.cc:2749] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:57.253683 25601 leader_election.cc:304] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:57.253952 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e41fb20a5d340ab95f58b3ccc5c680f" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:57.254813 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78bc05d6a56b407bb7507c25a641dc4e" candidate_uuid: "0ae979356d694a09be4b0cd976ea354a" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:57.255466 24421 tablet_replica.cc:333] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.256579 24421 raft_consensus.cc:2243] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.261806 24421 raft_consensus.cc:2272] T 294a99dd73c8449f84538cf8965a6467 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.272433 24421 tablet_replica.cc:333] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.273499 24421 raft_consensus.cc:2243] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.274094 24421 raft_consensus.cc:2272] T 3a039e6df55747e1b662ecb5d069264b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.276916 24421 tablet_replica.cc:333] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.277581 24421 raft_consensus.cc:2243] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.278363 24421 raft_consensus.cc:2272] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.278723   997 raft_consensus.cc:2706] T f80a8201dcaf4f2a9290c47595f9cf7b P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Received pre-election callback for term 21 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20260709 12:20:57.282071 24421 tablet_replica.cc:333] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.286586 24421 raft_consensus.cc:2243] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.287180 24421 raft_consensus.cc:2272] T 8c30f8fbe9ff44588b4847921bab9bd0 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.290669 24421 tablet_replica.cc:333] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.291906 24421 raft_consensus.cc:2243] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.293558 24421 raft_consensus.cc:2272] T b1ee3e27f67449aaab6c88508697a762 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.272359 25601 leader_election.cc:304] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.295636   997 raft_consensus.cc:2749] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:57.296310 24421 tablet_replica.cc:333] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.295737 25601 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.297302 24421 raft_consensus.cc:2243] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.298471 24421 raft_consensus.cc:2272] T bfd362b7abdb43de96e8b4d518fbb0f9 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.301295 24421 tablet_replica.cc:333] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.302148 24421 raft_consensus.cc:2243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.302489   997 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:57.303174 24421 raft_consensus.cc:2272] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.304857 25601 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.306206 24421 tablet_replica.cc:333] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.307219  1008 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:57.312285 26337 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:57.319360 24421 raft_consensus.cc:2243] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.320477 24421 raft_consensus.cc:2272] T c5dd00c486ca4b45b83fda5cf38079e8 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.323763 24421 tablet_replica.cc:333] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.324378   997 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:57.326022 25601 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.328302   997 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:57.329847 25601 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:57.330592 25601 raft_consensus.cc:2686] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 0ae979356d694a09be4b0cd976ea354a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:57.331449 25601 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.332576 24421 raft_consensus.cc:2243] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.333461   997 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:57.334975 24421 raft_consensus.cc:2272] T b4174bd2da7548489ad8aa892dabb8bb P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.347568 24421 tablet_replica.cc:333] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.350399 24421 raft_consensus.cc:2243] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.351200 24421 raft_consensus.cc:2272] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.349237 25601 leader_election.cc:304] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:57.353377  1004 raft_consensus.cc:2749] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:57.353700 25601 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.355558  1004 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:57.354956 25601 leader_election.cc:304] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:57.360546   997 raft_consensus.cc:2749] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:57.359427 25782 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:57.359306 24421 tablet_replica.cc:333] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.363581 24421 raft_consensus.cc:2243] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.364667 24421 raft_consensus.cc:2272] T f6cc82bb18354fbf8395678374e211b2 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.367489 24421 tablet_replica.cc:333] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.372574 25601 leader_election.cc:304] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae979356d694a09be4b0cd976ea354a; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:20:57.373394 25601 raft_consensus.cc:2686] T 7e41fb20a5d340ab95f58b3ccc5c680f P 0ae979356d694a09be4b0cd976ea354a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:57.373001 25531 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: config changed from index 10 to 11, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:57.375200 24421 raft_consensus.cc:2243] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.383140 24421 raft_consensus.cc:2272] T 6158b789bb894b2b9ad0c019dde5476f P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.395892 24421 tablet_replica.cc:333] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.397006 24421 raft_consensus.cc:2243] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.398691 24421 raft_consensus.cc:2272] T 60051e5723b2429582407076a003096c P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.403275 24421 tablet_replica.cc:333] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.404189 24421 raft_consensus.cc:2243] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.405102 24421 raft_consensus.cc:2272] T 6eb70fbead204d9581ca707c05f2befe P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.408670 24421 tablet_replica.cc:333] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.409595 24421 raft_consensus.cc:2243] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.410256 24421 raft_consensus.cc:2272] T c2f7c369dcd245eeba8804ccb208c79a P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.414004 24421 tablet_replica.cc:333] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.414970 24421 raft_consensus.cc:2243] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.417793 24421 raft_consensus.cc:2272] T b06b96578363435cbde83279e86e5c86 P 0ae979356d694a09be4b0cd976ea354a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.421180 24421 tablet_replica.cc:333] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.422318 24421 raft_consensus.cc:2243] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.423211 24421 raft_consensus.cc:2272] T b9052f9b88ca4b9ebb24b8764ad345af P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.426702 24421 tablet_replica.cc:333] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.427583 24421 raft_consensus.cc:2243] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.432276 24421 raft_consensus.cc:2272] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.444243 24421 tablet_replica.cc:333] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.445176 24421 raft_consensus.cc:2243] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.446587 24421 raft_consensus.cc:2272] T 2c9b915f0d524c19b87995309e7ea872 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.454519 24421 tablet_replica.cc:333] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.455401 24421 raft_consensus.cc:2243] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.463560 24421 raft_consensus.cc:2272] T eb89b7cd7f4d4662ab9d1e1f55af650c P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.468027 24421 tablet_replica.cc:333] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.468961 24421 raft_consensus.cc:2243] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.469703 24421 raft_consensus.cc:2272] T e905093d8adb4c6797cd77172a30843c P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.486847 24421 tablet_replica.cc:333] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.503297 24421 raft_consensus.cc:2243] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.504244 24421 raft_consensus.cc:2272] T c3c8e03aa6454e118cd3f9f25c785bef P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.541882 24421 tablet_replica.cc:333] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.542812 24421 raft_consensus.cc:2243] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.544076 24421 raft_consensus.cc:2272] T b5e8bc4d29af4139b4cb0e20fc57b10a P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.577212 24421 tablet_replica.cc:333] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.578217 24421 raft_consensus.cc:2243] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.579560 24421 raft_consensus.cc:2272] T fb1874a5bbce42ad864b2ba3624a0c1c P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.586683 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76504.2us
I20260709 12:20:57.594053 25708 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:57.604324 26337 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:57.607788 25782 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:57.613200   996 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:57.614419   990 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:57.619258 24421 tablet_replica.cc:333] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.620383 24421 raft_consensus.cc:2243] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.621135 24421 raft_consensus.cc:2272] T 2a18426c9efd45e698b2b2ccb80a49e8 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.632714  1008 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 11 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:57.655309 26335 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:57.657367 25782 raft_consensus.cc:2955] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:57.672497 24421 tablet_replica.cc:333] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.673436 24421 raft_consensus.cc:2243] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.674146 24421 raft_consensus.cc:2272] T 8a1f6623a51a47759e59269cfa63be4b P 0ae979356d694a09be4b0cd976ea354a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.678846 25522 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e905093d8adb4c6797cd77172a30843c with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:57.687392 25533 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 11 to 12, VOTER 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) evicted. New cstate: current_term: 14 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:20:57.693054 24421 tablet_replica.cc:333] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.694044 24421 raft_consensus.cc:2243] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.694684 24421 raft_consensus.cc:2272] T a741c75d1e684375b51f415fd0aa0296 P 0ae979356d694a09be4b0cd976ea354a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.697935 24421 tablet_replica.cc:333] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.722951 24421 raft_consensus.cc:2243] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.736034 24421 raft_consensus.cc:2272] T e13025a828f141378027526ae4f0b002 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.739702 24421 tablet_replica.cc:333] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.740521 24421 raft_consensus.cc:2243] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.741214 24421 raft_consensus.cc:2272] T 88f6873460634e2eb3253677a6a6b857 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.746320 24421 tablet_replica.cc:333] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.753964 24421 raft_consensus.cc:2243] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.754824 24421 raft_consensus.cc:2272] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.757820 24421 tablet_replica.cc:333] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.764647 24421 raft_consensus.cc:2243] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.765847 24421 raft_consensus.cc:2272] T e5735f590eff4fc9ae2686d09f8b6810 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.768715 24421 tablet_replica.cc:333] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.769603 24421 raft_consensus.cc:2243] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.770318 24421 raft_consensus.cc:2272] T 9d711fa725d244128d79dfdfb1e70af6 P 0ae979356d694a09be4b0cd976ea354a [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.772890 25521 catalog_manager.cc:4755] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e905093d8adb4c6797cd77172a30843c: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 12:20:57.808655 24421 tablet_replica.cc:333] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.809687 24421 raft_consensus.cc:2243] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.813047 24421 raft_consensus.cc:2272] T afe9ac51e0a949a7a43e288f3d2ed652 P 0ae979356d694a09be4b0cd976ea354a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.827070 24421 tablet_replica.cc:333] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.828058 24421 raft_consensus.cc:2243] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.829249 24421 raft_consensus.cc:2272] T 402bc370028b4bc097d5c2fb4342cdfa P 0ae979356d694a09be4b0cd976ea354a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.831944 24421 tablet_replica.cc:333] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.832708 24421 raft_consensus.cc:2243] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.833436 24421 raft_consensus.cc:2272] T 571c5e0767064f3e86479b794294d327 P 0ae979356d694a09be4b0cd976ea354a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.849092 24421 tablet_replica.cc:333] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.849957 24421 raft_consensus.cc:2243] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.854126 24421 raft_consensus.cc:2272] T 85b6713c7d844fb7a5272cac45763c0a P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.858197 24421 tablet_replica.cc:333] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.868561 24421 raft_consensus.cc:2243] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.883808 24421 raft_consensus.cc:2272] T b74717dca3554e889066a717cfd1527a P 0ae979356d694a09be4b0cd976ea354a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.892124 24421 tablet_replica.cc:333] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.892971 24421 raft_consensus.cc:2243] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.906126 24421 raft_consensus.cc:2272] T cb52f5946670453a8ec9b21d12ececbb P 0ae979356d694a09be4b0cd976ea354a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.909230 24421 tablet_replica.cc:333] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.910022 24421 raft_consensus.cc:2243] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.915326 24421 raft_consensus.cc:2272] T e0c8da68209b48f8868677e30067d396 P 0ae979356d694a09be4b0cd976ea354a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.933111 24421 tablet_replica.cc:333] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.933971 24421 raft_consensus.cc:2243] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.935211 24421 raft_consensus.cc:2272] T 4ff8dbae494348a7be4563e3bd2982b6 P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.943602 24421 tablet_replica.cc:333] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.944322 24421 raft_consensus.cc:2243] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.944903 24421 raft_consensus.cc:2272] T 78bc05d6a56b407bb7507c25a641dc4e P 0ae979356d694a09be4b0cd976ea354a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.947621 24421 tablet_replica.cc:333] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a: stopping tablet replica
I20260709 12:20:57.948248 24421 raft_consensus.cc:2243] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.948798 24421 raft_consensus.cc:2272] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 0ae979356d694a09be4b0cd976ea354a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.481698   964 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.482326   964 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:58.484670   964 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:58.486898 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:58.505750 25751 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.506465 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:58.512917  1000 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:58.537747   964 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.538340   964 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.540851   964 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:58.543349 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:20:58.547432 25751 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.547886 25751 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:58.549201  1000 raft_consensus.cc:2749] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:58.562498 25677 consensus_peers.cc:597] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:58.581040   957 tablet_copy_client.cc:1131] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.607811  1008 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.608600  1008 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:58.607940   995 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.612201   995 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:58.613224 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.613930 25780 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
I20260709 12:20:58.614656   995 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.616642 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:58.617331 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.619203  1008 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.619818  1008 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:58.622203  1008 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.624033  1008 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:58.626190 25677 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:58.627691 25677 leader_election.cc:336] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.628175 25677 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:58.628656   977 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.629376   977 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.632138   995 raft_consensus.cc:2749] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:58.632954   995 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.633952   977 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.633570   995 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:58.637048 26308 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.638296 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.648550   964 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.648825 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:58.649246   964 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.651190 25676 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348
I20260709 12:20:58.652266   964 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:58.653182   995 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:58.655795   995 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:58.657238 25751 leader_election.cc:336] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:58.658241 25677 leader_election.cc:336] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:58.659926 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 44937.1us
I20260709 12:20:58.661428 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.662074 25780 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:58.663318 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:58.666668  1005 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.667353  1005 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:58.669361 26307 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:58.669409   995 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.671945  1021 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.671770   995 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.672330 25750 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:58.672537  1021 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:58.694047   984 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:58.694643   984 raft_consensus.cc:493] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.695088   984 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:58.699666  1005 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:58.701261  1021 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:58.701467 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:58.675729  1000 raft_consensus.cc:2749] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:58.675868   964 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.684727   995 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:58.702977  1000 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.703552  1000 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.706179  1000 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:58.706648  1021 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.710695   964 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:58.711314  1021 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:58.713867   964 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:58.709626  1008 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:20:58.707533 25751 leader_election.cc:336] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:58.723443 25751 leader_election.cc:336] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.724370  1021 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.753120 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.753805 25782 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:58.758127   977 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.759613   977 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.766932 26337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:58.767369  1008 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.774886 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.774945 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:58.777207 25780 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:58.775847 25782 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
I20260709 12:20:58.775786 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:58.779188 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 103823us
I20260709 12:20:58.786085  1021 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:58.786603  1021 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.787015  1021 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:58.778148  1023 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:58.799206 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 114049us
I20260709 12:20:58.802978  1021 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:58.805521 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:58.803444 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:58.806282 25708 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
I20260709 12:20:58.809020 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:58.809623 25708 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:58.798406   977 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:58.803678  1031 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.835047  1031 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.836707 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:58.837157 25780 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 22.
I20260709 12:20:58.837621 25708 raft_consensus.cc:2393] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 21: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
W20260709 12:20:58.838902 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44581.6us
I20260709 12:20:58.839985 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:58.800099  1023 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:58.842231  1029 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:58.842783  1029 raft_consensus.cc:493] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.843168  1029 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:58.846933  1030 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.847505  1030 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:58.854714 25677 leader_election.cc:336] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.856622 25677 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:58.858222   995 raft_consensus.cc:2749] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:20:58.860002 25677 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.864401  1008 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:58.864919  1008 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.865350  1008 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:58.870005 25677 leader_election.cc:336] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:58.871345 25677 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:58.872668 25677 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.873417   977 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.874179   977 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:58.879043 25677 leader_election.cc:336] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.879331  1023 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:58.880333 26308 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.886382   977 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:58.887503 25751 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:58.887519 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:58.888283 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:58.888654  1030 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:58.888768 26308 leader_election.cc:336] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.887713  1022 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.852357   984 raft_consensus.cc:515] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.838300  1031 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.895865 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:58.896674   984 leader_election.cc:290] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.896865 25708 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:20:58.889976  1022 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.890455  1030 raft_consensus.cc:3060] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:58.898732 25752 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
W20260709 12:20:58.903429 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 100064us
I20260709 12:20:58.904132  1005 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:58.904640  1005 raft_consensus.cc:493] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.905057  1005 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:58.905195 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.905648  1021 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:58.907716 25677 consensus_peers.cc:597] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:58.909034 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:58.909746 26335 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
I20260709 12:20:58.913265 25787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.913924 25787 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 21.
I20260709 12:20:58.914042 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9b915f0d524c19b87995309e7ea872" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:58.914834 25779 raft_consensus.cc:3060] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:58.915723 25750 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:58.918519  1024 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:58.918926  1024 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.919297  1024 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:58.922533 25677 consensus_peers.cc:597] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:58.923667 25779 raft_consensus.cc:2468] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 17.
W20260709 12:20:58.927906 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:58.928431 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:58.930774 25751 leader_election.cc:336] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.931841  1022 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:58.932217 26307 leader_election.cc:304] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:20:58.928015  1024 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.933836  1032 raft_consensus.cc:2804] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:58.942080  1032 raft_consensus.cc:697] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:58.942795  1028 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.943385  1028 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.943204  1032 consensus_queue.cc:237] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:58.946398  1028 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:58.947762  1028 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:58.948329  1028 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:58.958155 26308 leader_election.cc:336] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.959024 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
W20260709 12:20:58.959964 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:58.960114 25708 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:58.962262  1028 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:58.963917 25677 leader_election.cc:336] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.964308 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:58.965510 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:58.967384 25708 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 21.
I20260709 12:20:58.968969 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:58.969920 25779 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:58.971414 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:20:58.972628 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:58.973102 26308 leader_election.cc:336] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:58.967738 25704 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 13.
I20260709 12:20:58.978578 25676 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:58.984097 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e8bc4d29af4139b4cb0e20fc57b10a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:58.986852 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:58.986935 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:58.989089 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 52390.9us
I20260709 12:20:58.991282 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:58.992715 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:58.981755  1027 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:58.982738   995 raft_consensus.cc:2804] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:59.008093  1024 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:58.979415  1029 raft_consensus.cc:515] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:58.994441   977 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:59.019861  1033 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:59.023121  1033 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.023658  1033 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:59.030014  1029 leader_election.cc:290] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.004730  1027 raft_consensus.cc:493] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.020912   977 raft_consensus.cc:493] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.061712   977 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:59.017925   995 raft_consensus.cc:697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:59.027820  1030 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:59.063936 26308 leader_election.cc:336] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:59.065702 25677 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.059113  1027 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:59.076983  1005 raft_consensus.cc:515] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.078464 25780 raft_consensus.cc:2393] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:59.062880  1023 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
W20260709 12:20:59.112286 25521 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 102037us
W20260709 12:20:59.113312 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 99200.2us
I20260709 12:20:59.115844 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.116514 25704 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:59.115101  1023 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.058682  1008 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:59.121624 25779 raft_consensus.cc:2468] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
I20260709 12:20:59.124444  1023 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:20:59.125797 25521 catalog_manager.cc:4755] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e905093d8adb4c6797cd77172a30843c: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.136341  1005 leader_election.cc:290] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.142452  1034 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.145254  1034 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.148243  1034 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:59.144183   995 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:59.159190   977 raft_consensus.cc:515] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.161237 26335 raft_consensus.cc:2393] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 16: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
W20260709 12:20:59.163151 25677 consensus_peers.cc:597] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:59.163726 25677 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:59.164474 25677 consensus_peers.cc:597] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:59.164906 25677 consensus_peers.cc:597] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.165766   977 leader_election.cc:290] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.142465  1008 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:59.178715  1041 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.179418  1041 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:59.183074 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:59.183707 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:59.184150 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.182250  1041 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:59.190385 26308 leader_election.cc:336] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.196870  1008 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.197202  1033 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.197597  1008 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:59.204068 26308 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.204456  1008 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:59.201434 25677 leader_election.cc:336] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:59.205257 25677 consensus_peers.cc:597] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:59.205749 25677 leader_election.cc:336] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.221194 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.217940   977 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.252646   977 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.255003   977 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.257995  1041 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:59.258574  1041 raft_consensus.cc:493] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.259035  1041 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:59.230638 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6ac1825a4e4ab9bb0d7d2867816cbe" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:59.244020 25677 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.261297 25709 raft_consensus.cc:3060] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:59.216113 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dd00c486ca4b45b83fda5cf38079e8" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.267519 25708 raft_consensus.cc:3060] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:59.245476  1036 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.251179 25533 catalog_manager.cc:5697] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 16 to 17, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 17 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:59.267282 25751 leader_election.cc:336] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:59.271827 25751 leader_election.cc:336] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.271579  1036 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:59.272326 25751 leader_election.cc:336] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:59.209631 25750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 219724us
I20260709 12:20:59.224115 25704 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
I20260709 12:20:59.278380 26335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:59.279165 26335 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 22: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:59.297441  1036 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.300642  1033 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:20:59.301396 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.276688 25676 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:59.305840 26308 leader_election.cc:336] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.290299 25709 raft_consensus.cc:2468] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
W20260709 12:20:59.308663 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.309890  1027 raft_consensus.cc:515] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.315209  1023 raft_consensus.cc:2749] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:59.275195  1041 raft_consensus.cc:515] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:59.313789 25751 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.316936 25676 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:20:59.323876 24421 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 25659)
I20260709 12:20:59.325601 25750 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:59.328784 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:59.329569 26342 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 12.
I20260709 12:20:59.317543  1041 leader_election.cc:290] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.321635  1027 leader_election.cc:290] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:59.322364 25708 raft_consensus.cc:2468] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:59.335443 25752 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.335549 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:20:59.335697  1023 raft_consensus.cc:3060] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:59.336098  1038 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:20:59.356820 25677 consensus_peers.cc:597] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.362846  1038 raft_consensus.cc:2804] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:59.363579  1038 raft_consensus.cc:697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
W20260709 12:20:59.363694 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.364651  1038 consensus_queue.cc:237] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.370754 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:59.371549 25779 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:59.377122  1008 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.377772  1008 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.380098  1008 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:59.381155 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1874a5bbce42ad864b2ba3624a0c1c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
W20260709 12:20:59.385897 25677 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.387897 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:59.388852 25780 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 22: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
W20260709 12:20:59.375149 26308 leader_election.cc:336] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.388818 25779 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 22.
I20260709 12:20:59.393854 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:20:59.399271 26308 leader_election.cc:336] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.394554 25779 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 17.
W20260709 12:20:59.399943 26308 leader_election.cc:336] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.402750 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:59.406165 25528 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 21 to 22, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 22 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:59.409171 25704 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 12.
I20260709 12:20:59.414973  1036 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.415555  1036 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.416816  1027 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.417481  1027 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.419940 25782 raft_consensus.cc:1275] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:59.427151 25707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:59.432862 25707 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 17.
I20260709 12:20:59.421093 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6cc82bb18354fbf8395678374e211b2" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:59.445241 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:59.439517  1023 raft_consensus.cc:2749] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:59.441085 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b06b96578363435cbde83279e86e5c86" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.464572 25706 raft_consensus.cc:3060] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:59.440112 25704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:59.467621 26307 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:59.440788 25709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.468706 25709 raft_consensus.cc:2393] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 22: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:59.468675  1033 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:59.463871 25711 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:59.448880 25676 leader_election.cc:304] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:59.472178 25676 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.467862 25704 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 19.
I20260709 12:20:59.472609 25708 raft_consensus.cc:2468] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 16.
I20260709 12:20:59.473001   995 raft_consensus.cc:2749] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:59.473873   995 raft_consensus.cc:2804] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:59.439785  1027 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.474387   995 raft_consensus.cc:697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
W20260709 12:20:59.438529 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.476117   995 consensus_queue.cc:237] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.478272 25706 raft_consensus.cc:2468] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
I20260709 12:20:59.478672 25752 leader_election.cc:304] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:59.474396 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.462224 25779 raft_consensus.cc:3060] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:59.487066 25676 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.438668  1036 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.491488  1023 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:59.492144  1023 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:20:59.494935 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.496994  1028 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:59.497557  1028 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.497999  1028 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:59.493402  1023 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:20:59.500025 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.500895  1045 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:59.501284  1045 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.501650  1045 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:59.511166 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:20:59.512431 25706 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 20.
I20260709 12:20:59.508590 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
W20260709 12:20:59.522075 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.522692 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.523617  1008 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:59.524044  1008 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.524518  1008 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:59.508872 25778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:20:59.529430 25778 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 20.
I20260709 12:20:59.529879 25676 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:59.531744  1041 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:59.532275  1041 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.532676  1041 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:20:59.530519 26308 leader_election.cc:336] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.534257 25779 raft_consensus.cc:2468] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:59.488483  1038 raft_consensus.cc:2804] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:59.537228 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.537781 26307 leader_election.cc:304] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.541882 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.543402   977 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:59.543841   977 raft_consensus.cc:493] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.544209   977 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:59.548828  1042 raft_consensus.cc:2749] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:59.550452 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.550762 25531 catalog_manager.cc:5697] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 15 to 16, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 16 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:59.535616  1027 consensus_queue.cc:1048] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:20:59.559302  1031 raft_consensus.cc:2804] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:59.565789 26309 leader_election.cc:304] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.566083  1046 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:59.583043  1031 raft_consensus.cc:697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:59.583436  1046 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.583941  1046 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:59.585772 25677 consensus_peers.cc:597] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:59.586457 25677 consensus_peers.cc:597] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.586211  1031 consensus_queue.cc:237] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.587973 26309 leader_election.cc:304] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.586715  1022 raft_consensus.cc:2804] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:20:59.591308 25677 consensus_peers.cc:597] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.578526  1028 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.556972 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.596115  1042 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:59.596576  1042 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.569187  1008 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.596959 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:59.594267  1028 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:59.597016  1042 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:59.599020  1008 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:20:59.602936  1022 raft_consensus.cc:697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:59.604010  1022 consensus_queue.cc:237] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.607899 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:20:59.608120  1046 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.602982  1043 raft_consensus.cc:2804] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:59.610414  1043 raft_consensus.cc:697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:59.610706  1046 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:59.611683 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61343.2us
I20260709 12:20:59.551563  1038 raft_consensus.cc:697] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:20:59.617635  1038 consensus_queue.cc:237] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.619330  1043 consensus_queue.cc:237] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.619844 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.620493 25708 raft_consensus.cc:3060] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:59.542518  1045 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:59.614893 25677 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.616482  1036 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:59.631780 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:59.632153  1045 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.632506 25779 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 18: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:59.636479 25708 raft_consensus.cc:2468] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 20.
I20260709 12:20:59.646100 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:59.650580 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.653810 26309 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:20:59.647214  1023 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:59.632525  1036 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0ae979356d694a09be4b0cd976ea354a)
I20260709 12:20:59.648772  1041 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.681097  1036 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:59.681653 26342 raft_consensus.cc:2393] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 13: Already voted for candidate 10d0c7349085463595aedaf7c071e742 in this term.
I20260709 12:20:59.682685  1041 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.684994 25706 raft_consensus.cc:2393] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 18: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:59.663456   977 raft_consensus.cc:515] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:20:59.671978 25677 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:59.675028 25751 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:20:59.687348 26308 leader_election.cc:336] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.696090 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.696956 25708 raft_consensus.cc:2393] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 10d0c7349085463595aedaf7c071e742 in current term 13: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:59.692408  1041 raft_consensus.cc:2804] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:59.678364  1042 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:20:59.712579 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 44520us
W20260709 12:20:59.724560 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.710935  1041 raft_consensus.cc:697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:59.720821   977 leader_election.cc:290] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:20:59.728145 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:59.729454 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:20:59.732396 26342 raft_consensus.cc:3060] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Advancing to term 22
I20260709 12:20:59.734571  1038 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:59.741178  1042 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:20:59.744310 25782 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:20:59.752236 25677 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.755538  1008 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:59.758534 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff8dbae494348a7be4563e3bd2982b6" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.762526 25706 raft_consensus.cc:3060] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:59.762759 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.768867 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:20:59.770344 26342 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:20:59.773521 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:59.774165 26308 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.774681 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:59.774407  1041 consensus_queue.cc:237] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.775653  1034 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:20:59.781122 25677 consensus_peers.cc:597] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:59.785147 26308 leader_election.cc:336] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.784255  1043 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:59.795704  1028 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:59.799576  1036 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:20:59.806970 25706 raft_consensus.cc:2468] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:20:59.811110 26309 leader_election.cc:304] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:20:59.826992  1043 raft_consensus.cc:2804] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:59.827697  1043 raft_consensus.cc:697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:20:59.830785 25782 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 21: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:20:59.836109  1043 consensus_queue.cc:237] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:20:59.864200 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:20:59.865242 25706 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 21: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:20:59.864671 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:20:59.873822  1008 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:59.846285  1036 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:20:59.896059 25677 consensus_peers.cc:597] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:59.949051 25780 raft_consensus.cc:1275] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:20:59.950549 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:20:59.951020 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:20:59.961527  1045 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:59.970165 25531 catalog_manager.cc:5697] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 16 to 17, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 17 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:59.981813 25533 catalog_manager.cc:5697] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 20 to 21, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 21 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:59.983942 25533 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 21 to 22, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 22 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:59.991926 25528 catalog_manager.cc:5697] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 13 to 14, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 14 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:59.994249 25528 catalog_manager.cc:5697] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 19 to 20, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:59.995939 25528 catalog_manager.cc:5697] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 15 to 16, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 16 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.000070 25708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35600
I20260709 12:21:00.000826 25708 raft_consensus.cc:606] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Received request to transfer leadership
I20260709 12:21:00.007952 25706 raft_consensus.cc:1275] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:21:00.009759 25751 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.015533  1005 consensus_queue.cc:1048] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:21:00.015650  1028 consensus_queue.cc:1048] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
W20260709 12:21:00.021348 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.000134 25528 catalog_manager.cc:5697] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 19 to 20, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 20 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.032958 25528 catalog_manager.cc:5697] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 14 to 15, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 15 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.043244 25706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35600
I20260709 12:21:00.044108 25706 raft_consensus.cc:606] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Received request to transfer leadership
W20260709 12:21:00.045976 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.084256 25784 raft_consensus.cc:1275] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:21:00.100685 25706 raft_consensus.cc:1275] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:21:00.102720  1045 consensus_queue.cc:1048] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:21:00.092952  1033 consensus_queue.cc:1048] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:21:00.122740 25708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467"
dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:35600
I20260709 12:21:00.123620 25708 raft_consensus.cc:606] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Received request to transfer leadership
W20260709 12:21:00.139272 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:00.134023 25677 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:00.142441 25677 consensus_peers.cc:597] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:00.142994 25677 consensus_peers.cc:597] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:00.149817 25584 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:21:00.150254 25584 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 294a99dd73c8449f84538cf8965a6467 already in progress
W20260709 12:21:00.150581 25584 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 294a99dd73c8449f84538cf8965a6467 already in progress
W20260709 12:21:00.175293 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 101826us
I20260709 12:21:00.180289 26340 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 21 LEARNER]: Advancing to term 22
W20260709 12:21:00.195025 25751 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.217806   995 raft_consensus.cc:1064] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:21:00.220992   995 consensus_queue.cc:237] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 16, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:00.226086  1008 raft_consensus.cc:1064] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
I20260709 12:21:00.230918 26340 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 22 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:21:00.237547 25521 catalog_manager.cc:4755] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e905093d8adb4c6797cd77172a30843c: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:00.244481 25708 raft_consensus.cc:1275] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 12:21:00.245651 25677 consensus_peers.cc:597] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.246426 25708 raft_consensus.cc:1275] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:21:00.251654 26340 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 LEARNER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:21:00.254001 25784 raft_consensus.cc:1275] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:21:00.256705 25784 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:21:00.256863  1008 raft_consensus.cc:1068] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: Unable to promote non-voter 10d0c7349085463595aedaf7c071e742: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }.
  Pending config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:00.273810  1046 consensus_queue.cc:1048] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:00.276337  1046 consensus_queue.cc:1048] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:21:00.280400  1077 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:21:00.284127   995 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:21:00.290956 26308 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.310930  1077 consensus_queue.cc:1048] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:21:00.316012   995 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:21:00.324530 24421 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 25659)
W20260709 12:21:00.331687 26308 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.383294 25704 raft_consensus.cc:1275] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:21:00.389555   995 raft_consensus.cc:2955] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Committing config change with OpId 22.18: config changed from index 16 to 18, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:21:00.404100  1043 consensus_queue.cc:1048] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
W20260709 12:21:00.420068 26308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111) [suppressed 306 similar messages]
I20260709 12:21:00.482883 25780 raft_consensus.cc:2955] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Committing config change with OpId 22.18: config changed from index 16 to 18, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:21:00.514456  1077 raft_consensus.cc:1064] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
W20260709 12:21:00.514957  1077 raft_consensus.cc:1068] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Unable to promote non-voter 10d0c7349085463595aedaf7c071e742: Service unavailable: leader transfer in progress
I20260709 12:21:00.516589  1008 raft_consensus.cc:993] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: : Instructing follower 955233c139344131a847044f17f8d348 to start an election
I20260709 12:21:00.517047  1008 raft_consensus.cc:1081] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Signalling peer 955233c139344131a847044f17f8d348 to start an election
I20260709 12:21:00.535573 25784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467"
dest_uuid: "955233c139344131a847044f17f8d348"
 from {username='slave'} at 127.0.0.1:50966
I20260709 12:21:00.536204 25784 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:00.536527 25784 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.549124 25528 catalog_manager.cc:5697] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 16 to 18, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:21:00.570422 25706 raft_consensus.cc:1275] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:21:00.547367 25784 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:21:00.576529 26340 raft_consensus.cc:2955] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Committing config change with OpId 22.18: config changed from index 16 to 18, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
W20260709 12:21:00.578125 25751 leader_election.cc:336] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:00.585565 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294a99dd73c8449f84538cf8965a6467" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:21:00.587213 25706 raft_consensus.cc:3055] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:21:00.587527 25706 raft_consensus.cc:740] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:21:00.588074 25784 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:00.588696  1022 consensus_queue.cc:1048] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:21:00.644065 25706 consensus_queue.cc:260] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
I20260709 12:21:00.648272 25706 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.672310 25706 raft_consensus.cc:2468] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 23.
I20260709 12:21:00.675619  1042 raft_consensus.cc:1064] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
W20260709 12:21:00.676870  1042 raft_consensus.cc:1068] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: Unable to promote non-voter 10d0c7349085463595aedaf7c071e742: Illegal state: Replica c7bf8f20701245bcbaa09cc663025f97 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } } }
I20260709 12:21:00.682221 25752 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:00.683790  1045 raft_consensus.cc:2804] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:00.684302  1045 raft_consensus.cc:697] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 23 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:21:00.685324  1045 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } }
W20260709 12:21:00.749226 25677 consensus_peers.cc:597] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:00.753299 25531 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 22 to 23, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 23 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:01.124475 25751 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:01.142812 25706 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:21:01.144640  1084 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:01.176215  1033 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.176873  1033 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:01.181243 26340 raft_consensus.cc:3060] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 22 LEARNER]: Advancing to term 23
I20260709 12:21:01.187394  1033 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:01.188108 25706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:21:01.188819 25706 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 22.
I20260709 12:21:01.202425 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:21:01.203203 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 22.
I20260709 12:21:01.206399 26307 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:21:01.208213  1033 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:01.208853  1033 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.209223  1033 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:21:01.217300 26308 leader_election.cc:336] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:01.226227 26340 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:21:01.229732  1045 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: NON_VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:21:01.255893 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.272305 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.278189  1033 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:01.289789  1033 leader_election.cc:290] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:01.291775 25711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:21:01.292555 25711 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:01.293010 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:21:01.293664 25784 raft_consensus.cc:3060] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:01.305382 25784 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 23.
I20260709 12:21:01.309772  1036 raft_consensus.cc:1064] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: attempting to promote NON_VOTER 10d0c7349085463595aedaf7c071e742 to VOTER
W20260709 12:21:01.325109 24421 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 25659)
I20260709 12:21:01.324254  1036 consensus_queue.cc:237] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:01.339636 26340 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 LEARNER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:21:01.341132 25706 raft_consensus.cc:1275] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:21:01.343113  1045 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:01.350111  1084 consensus_queue.cc:1048] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:01.359390  1005 raft_consensus.cc:2955] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 13 to 16, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:21:01.362723 26340 raft_consensus.cc:2955] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 13 to 16, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:21:01.309754 25711 raft_consensus.cc:2468] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 23.
I20260709 12:21:01.375022 26309 leader_election.cc:304] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 
I20260709 12:21:01.376724  1033 raft_consensus.cc:2804] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:01.378907  1033 raft_consensus.cc:697] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 23 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:21:01.380215  1033 consensus_queue.cc:237] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:01.385639 25751 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:01.386242 25711 raft_consensus.cc:2955] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 13 to 16, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } } }
I20260709 12:21:01.393692 25528 catalog_manager.cc:5697] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 reported cstate change: config changed from index 13 to 16, 10d0c7349085463595aedaf7c071e742 (127.23.217.68) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:01.404896 26308 leader_election.cc:336] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:01.406039 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.406535 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.443385 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.455788  1075 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:01.471930 25677 consensus_peers.cc:597] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.472152  1075 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:01.480093 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.493454 25677 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:01.494733  1075 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:01.496799 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:21:01.497563 25784 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 18.
W20260709 12:21:01.499750 25677 consensus_peers.cc:597] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.500957 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.500910 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:01.502846  1042 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:21:01.503265  1042 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.503649  1042 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:21:01.505910 25533 catalog_manager.cc:5697] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 21 to 23, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 23 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:01.525564  1075 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.526239  1075 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:01.547295  1075 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:01.552583  1042 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:01.555214 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:21:01.555970 26342 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 13.
W20260709 12:21:01.557447 25677 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:01.558395 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:01.559213  1023 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:21:01.559629  1023 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.559978  1023 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:21:01.562961 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55838.7us
W20260709 12:21:01.565060 25677 consensus_peers.cc:597] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.579015 25677 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:01.586851 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.589598  1023 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:01.591878  1042 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:01.591969  1023 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:21:01.601204 25677 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:01.602414 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402bc370028b4bc097d5c2fb4342cdfa" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:21:01.603120 26342 raft_consensus.cc:3060] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:21:01.612224 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74717dca3554e889066a717cfd1527a" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:21:01.613127 25784 raft_consensus.cc:3060] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:21:01.612962  1042 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.623541  1042 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:01.634125 25784 raft_consensus.cc:2468] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 19.
I20260709 12:21:01.637288 25676 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:01.644045 26342 raft_consensus.cc:2468] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 14.
I20260709 12:21:01.644960  1005 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.645606  1005 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:01.648651  1005 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:01.649863  1042 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:01.651500  1042 raft_consensus.cc:2804] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:21:01.651993  1042 raft_consensus.cc:697] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:21:01.652879  1042 consensus_queue.cc:237] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:01.667307 25677 consensus_peers.cc:597] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.670284 25676 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:01.670652 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:21:01.671345 25784 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 21.
I20260709 12:21:01.677915  1023 raft_consensus.cc:2804] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:21:01.678824  1023 raft_consensus.cc:697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:21:01.680092  1023 consensus_queue.cc:237] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:01.688506 25677 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:01.710989 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97" is_pre_election: true
I20260709 12:21:01.711666 25708 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 21.
W20260709 12:21:01.713989 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:01.714975 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 44952.3us
W20260709 12:21:01.733635 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.734021 25677 consensus_peers.cc:597] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.759027 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.759770 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:01.763322  1075 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:01.763780  1075 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.764109  1075 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:21:01.781406 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 61541.5us
I20260709 12:21:01.782833 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:01.783771  1045 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:01.784170  1045 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.784502  1045 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:21:01.788125 25677 consensus_peers.cc:597] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:01.776358 25531 catalog_manager.cc:5697] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 17 to 19, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 19 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.796294  1075 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:01.793288  1045 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:01.803673 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.807690 25677 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:01.810981 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 51333.6us
W20260709 12:21:01.814877 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:01.817375 25708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c7bf8f20701245bcbaa09cc663025f97"
I20260709 12:21:01.818279 25708 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 955233c139344131a847044f17f8d348 in current term 22: Already voted for candidate c7bf8f20701245bcbaa09cc663025f97 in this term.
I20260709 12:21:01.812463  1045 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:01.820189 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:01.821470  1045 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:21:01.826064  1075 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:01.838927 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:21:01.840085 25784 raft_consensus.cc:2393] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c7bf8f20701245bcbaa09cc663025f97 in current term 22: Already voted for candidate 955233c139344131a847044f17f8d348 in this term.
I20260709 12:21:01.852512 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:21:01.856361  1075 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:21:01.893499 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.911281 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 58853.4us
I20260709 12:21:01.911566 25531 catalog_manager.cc:5697] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 12 to 14, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 14 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:01.939906 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:01.967058 26308 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:02.016229 25711 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:21:02.023926  1022 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:02.032058 25784 raft_consensus.cc:1275] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:21:02.033752   977 consensus_queue.cc:1048] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:21:02.061280 25677 consensus_peers.cc:597] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:02.079447 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:02.083204 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:02.158162 25784 raft_consensus.cc:1275] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:21:02.161643  1097 consensus_queue.cc:1048] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:21:02.164065 25677 consensus_peers.cc:597] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:02.170141 25677 consensus_peers.cc:597] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:02.171715 25677 consensus_peers.cc:597] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:02.233911 26340 raft_consensus.cc:1275] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:21:02.235860  1023 consensus_queue.cc:1048] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:21:02.249178 25677 consensus_peers.cc:597] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:02.299556 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:02.325683 24421 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 25659)
W20260709 12:21:02.337939 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:02.391184 25521 catalog_manager.cc:4755] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e905093d8adb4c6797cd77172a30843c: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:02.740907 25751 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:02.854100 26309 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 164409us
W20260709 12:21:02.886945 25677 consensus_peers.cc:597] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:03.009965 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 64457us
W20260709 12:21:03.020685 25677 consensus_peers.cc:597] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.079780 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 58196.8us
W20260709 12:21:03.089064 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 78482.7us
W20260709 12:21:03.121479 25677 consensus_peers.cc:597] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.128278 25677 consensus_peers.cc:597] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:03.131829 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.150313 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.151234 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.151870 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:03.167032 26308 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.324580 25677 consensus_peers.cc:597] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:03.326727 24421 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 25659)
W20260709 12:21:03.361838 25677 consensus_peers.cc:597] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:03.379444 25567 debug-util.cc:398] Leaking SignalData structure 0x7b080037d960 after lost signal to thread 24424
W20260709 12:21:03.381767 25567 debug-util.cc:398] Leaking SignalData structure 0x7b0800377fc0 after lost signal to thread 25570
W20260709 12:21:03.384680 25567 debug-util.cc:398] Leaking SignalData structure 0x7b080003e9a0 after lost signal to thread 25737
W20260709 12:21:03.386637 25567 debug-util.cc:398] Leaking SignalData structure 0x7b08001aabe0 after lost signal to thread 25811
W20260709 12:21:03.388996 25567 debug-util.cc:398] Leaking SignalData structure 0x7b08003bf860 after lost signal to thread 26368
W20260709 12:21:03.398415 25567 debug-util.cc:398] Leaking SignalData structure 0x7b0800140020 after lost signal to thread 1094
W20260709 12:21:03.398844 25567 debug-util.cc:398] Leaking SignalData structure 0x7b08001dec60 after lost signal to thread 1095
W20260709 12:21:03.441720 25751 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.444563 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 53533.4us
W20260709 12:21:03.690237 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:03.713429 26308 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.939050 25677 consensus_peers.cc:597] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.113549 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.171254 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.199774 25677 consensus_peers.cc:597] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.209403 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.225446 25677 consensus_peers.cc:597] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.267856 25677 consensus_peers.cc:597] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.285455 25751 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:04.316366 25677 consensus_peers.cc:597] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.327711 24421 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 25659)
W20260709 12:21:04.364044 25677 consensus_peers.cc:597] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.405126 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:04.454802  1123 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:04.455888  1123 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:04.461079  1123 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:04.490047 25784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:21:04.504226 25784 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 22.
W20260709 12:21:04.499931 25677 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:04.514050 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:04.522748 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:04.541539  1123 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:04.542186  1123 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:04.542636  1123 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:21:04.632026 25677 consensus_peers.cc:597] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:04.708127 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.752279 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.754853 25677 consensus_peers.cc:597] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:04.807150 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 52361.7us
W20260709 12:21:04.811367 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 59140.1us
W20260709 12:21:04.832226 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:04.834760  1123 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:04.911129  1123 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 election: Requested vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:04.856437 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 55011.7us
W20260709 12:21:04.937031 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.013504 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 76551.4us
W20260709 12:21:05.030876 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:35023, user_credentials={real_user=slave}} blocked reactor thread for 49809.7us
W20260709 12:21:04.964561 25677 consensus_peers.cc:597] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:04.994351 25779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0c8da68209b48f8868677e30067d396" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:21:05.037101 25779 raft_consensus.cc:3060] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:21:05.036890 25677 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 72372.5us
I20260709 12:21:05.049181 25779 raft_consensus.cc:2468] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7bf8f20701245bcbaa09cc663025f97 in term 23.
W20260709 12:21:05.057080 25677 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:05.057787 25677 consensus_peers.cc:597] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.146729 25677 consensus_peers.cc:597] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:05.159884 25751 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.163872 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.184971 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.185567 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.193181 26308 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:05.193670 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.218317 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.223330 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.226626 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:05.260772 25676 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a
I20260709 12:21:05.261979  1097 raft_consensus.cc:2804] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:05.262566  1097 raft_consensus.cc:697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 23 LEADER]: Becoming Leader. State: Replica: c7bf8f20701245bcbaa09cc663025f97, State: Running, Role: LEADER
I20260709 12:21:05.263751  1097 consensus_queue.cc:237] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:05.320667 25677 consensus_peers.cc:597] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.328743 24421 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 25659)
W20260709 12:21:05.338191 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.346359 25677 consensus_peers.cc:597] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:05.372911 25531 catalog_manager.cc:5697] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 reported cstate change: term changed from 20 to 23, leader changed from 0ae979356d694a09be4b0cd976ea354a (127.23.217.65) to c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66). New cstate: current_term: 23 leader_uuid: "c7bf8f20701245bcbaa09cc663025f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:05.422683 25677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111) [suppressed 408 similar messages]
W20260709 12:21:05.517948 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.557653 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.665366 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.719691 25677 consensus_peers.cc:597] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:05.748389 25677 consensus_peers.cc:597] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:05.756506 25677 consensus_peers.cc:597] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.809514 25677 consensus_peers.cc:597] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:05.869585 25780 raft_consensus.cc:1275] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Refusing update from remote peer c7bf8f20701245bcbaa09cc663025f97: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:21:05.877815  1123 consensus_queue.cc:1048] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:21:05.929032 25751 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:06.008338 25677 consensus_peers.cc:597] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.020838 26308 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:06.085704 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.329280 24421 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 25659)
W20260709 12:21:06.435119 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.481266 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.552129 25521 catalog_manager.cc:4755] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e905093d8adb4c6797cd77172a30843c: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:06.556018 25677 consensus_peers.cc:597] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.629418 25677 consensus_peers.cc:597] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.653210 25677 consensus_peers.cc:597] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.688360 25751 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:06.678136 26308 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:06.765450 25677 consensus_peers.cc:597] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.770480 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.912814 25677 consensus_peers.cc:597] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.916569 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:06.973526 25677 consensus_peers.cc:597] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.055414 25677 consensus_peers.cc:597] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.125772 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.208101 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.231050 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.243770 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.244664 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.256076 25677 consensus_peers.cc:597] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:07.320817 25751 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:07.323376 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.330307 24421 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 25659)
W20260709 12:21:07.341497 25677 consensus_peers.cc:597] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.379354 25677 consensus_peers.cc:597] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:07.400856 25677 consensus_peers.cc:597] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.413158 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.479908 25677 consensus_peers.cc:597] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.573537 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.576457 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:07.701040 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.701722 25677 consensus_peers.cc:597] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:07.722220 26308 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:07.723047 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.726197 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:07.838583 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:07.917090 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:07.965384 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:08.034886 25677 consensus_peers.cc:597] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:08.156937 25677 consensus_peers.cc:597] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:08.162113 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:08.201714 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
W20260709 12:21:08.263300 25751 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:08.339722 25677 consensus_peers.cc:597] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:08.435350 25677 consensus_peers.cc:597] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:08.443976 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:08.526057 26308 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:08.649441 25677 consensus_peers.cc:597] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:08.763001 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
W20260709 12:21:08.805294 25752 consensus_peers.cc:597] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.811667 26309 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.846642 26309 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.871512 26309 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.873324 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:08.877184 25752 consensus_peers.cc:597] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.878938 25677 consensus_peers.cc:597] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:08.888103 25677 consensus_peers.cc:597] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:08.900980 25752 consensus_peers.cc:597] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.916740 25752 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.938699 25752 consensus_peers.cc:597] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.968935 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:08.972808 24421 tablet_replica.cc:333] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:08.974349 24421 raft_consensus.cc:2243] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 12 LEADER]: Raft consensus shutting down.
W20260709 12:21:08.979949 25752 consensus_peers.cc:597] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.994843 24421 raft_consensus.cc:2272] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P c7bf8f20701245bcbaa09cc663025f97 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.998677 24421 tablet_replica.cc:333] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:08.999822 24421 raft_consensus.cc:2243] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.000860 24421 raft_consensus.cc:2272] T 12eaa1f2655a4cc7bde6bd409ccdf780 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.006657 24421 tablet_replica.cc:333] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.008508 24421 raft_consensus.cc:2243] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.013027 24421 raft_consensus.cc:2272] T 36db04d76d874d328966edd6be42e2c8 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.017916 25752 consensus_peers.cc:597] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.018939 24421 tablet_replica.cc:333] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.019965 24421 raft_consensus.cc:2243] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.021409 24421 raft_consensus.cc:2272] T 595303b3057546cbbfda4b84f8df8912 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.024734 24421 tablet_replica.cc:333] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.025695 24421 raft_consensus.cc:2243] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.028909 24421 raft_consensus.cc:2272] T 4ff8dbae494348a7be4563e3bd2982b6 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.035291 24421 tablet_replica.cc:333] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.036232 24421 raft_consensus.cc:2243] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.037755 24421 raft_consensus.cc:2272] T 294a99dd73c8449f84538cf8965a6467 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.042469 24421 tablet_replica.cc:333] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:21:09.047355 26309 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.050393 24421 raft_consensus.cc:2243] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.051254 24421 raft_consensus.cc:2272] T 5ac10fecbdc4482e93572bd7ff87319e P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.057523 26309 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.059331 24421 tablet_replica.cc:333] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:21:09.059943 25752 consensus_peers.cc:597] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.060415 24421 raft_consensus.cc:2243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.061879 24421 raft_consensus.cc:2272] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.064766 26309 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.066205 24421 tablet_replica.cc:333] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.080813 24421 raft_consensus.cc:2243] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.082906 24421 raft_consensus.cc:2272] T bfd362b7abdb43de96e8b4d518fbb0f9 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.086998 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:09.086773 26309 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.092602 24421 tablet_replica.cc:333] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.093552 24421 raft_consensus.cc:2243] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.094329 24421 raft_consensus.cc:2272] T 19825207f8744cf7809ab169c0ec8973 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.100919 25677 consensus_peers.cc:597] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:09.103760 25752 consensus_peers.cc:597] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.104425 24421 tablet_replica.cc:333] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.105545 24421 raft_consensus.cc:2243] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.107005 24421 raft_consensus.cc:2272] T 7e41fb20a5d340ab95f58b3ccc5c680f P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.109566 25752 consensus_peers.cc:597] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.111825 26309 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.123478 24421 tablet_replica.cc:333] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.124760 24421 raft_consensus.cc:2243] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.128342 24421 raft_consensus.cc:2272] T 6158b789bb894b2b9ad0c019dde5476f P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.131309 24421 tablet_replica.cc:333] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.132185 24421 raft_consensus.cc:2243] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.132992 24421 raft_consensus.cc:2272] T c5dd00c486ca4b45b83fda5cf38079e8 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.135908 24421 tablet_replica.cc:333] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:21:09.157546 26309 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.161733 25752 consensus_peers.cc:597] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.189898 25752 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.136950 24421 raft_consensus.cc:2243] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.197698 24421 raft_consensus.cc:2272] T b4174bd2da7548489ad8aa892dabb8bb P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.203217 25752 consensus_peers.cc:597] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.217698 26308 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:09.221627 24421 tablet_replica.cc:333] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.222743 24421 raft_consensus.cc:2243] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.229418 24421 raft_consensus.cc:2272] T 7701d2ef98204ad0bedc9231fe829f69 P c7bf8f20701245bcbaa09cc663025f97 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.236056 25752 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.254083 26309 consensus_peers.cc:597] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.263170 24421 tablet_replica.cc:333] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.264242 24421 raft_consensus.cc:2243] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.265290 24421 raft_consensus.cc:2272] T 9eec1cb847cc4587be1133c287c515b2 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.267970 25752 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.277746 24421 tablet_replica.cc:333] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.279011 24421 raft_consensus.cc:2243] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.279901 24421 raft_consensus.cc:2272] T b06b96578363435cbde83279e86e5c86 P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.310920 24421 tablet_replica.cc:333] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.311830 24421 raft_consensus.cc:2243] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.312531 24421 raft_consensus.cc:2272] T bd6ac1825a4e4ab9bb0d7d2867816cbe P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.336969 24421 tablet_replica.cc:333] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.337864 24421 raft_consensus.cc:2243] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.339193 24421 raft_consensus.cc:2272] T c2f7c369dcd245eeba8804ccb208c79a P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.342016 24421 tablet_replica.cc:333] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:21:09.362219 25677 consensus_peers.cc:597] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.362998 24421 raft_consensus.cc:2243] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.364403 24421 raft_consensus.cc:2272] T e5735f590eff4fc9ae2686d09f8b6810 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.367112 24421 tablet_replica.cc:333] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.367945 24421 raft_consensus.cc:2243] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.386534 24421 raft_consensus.cc:2272] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.409459 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:09.422351 25677 consensus_peers.cc:597] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:09.445572 25677 consensus_peers.cc:597] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.456337 24421 tablet_replica.cc:333] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.463948 24421 raft_consensus.cc:2243] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.464763 24421 raft_consensus.cc:2272] T 78bc05d6a56b407bb7507c25a641dc4e P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.544694 25751 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:09.556455 24421 tablet_replica.cc:333] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.557487 24421 raft_consensus.cc:2243] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.558458 24421 raft_consensus.cc:2272] T 6eb70fbead204d9581ca707c05f2befe P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.613232 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:09.615010 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.640723 24421 tablet_replica.cc:333] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.641728 24421 raft_consensus.cc:2243] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.657116 24421 raft_consensus.cc:2272] T 31ccbf41d24e4275a7f921db24b940d4 P c7bf8f20701245bcbaa09cc663025f97 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.674265 25677 consensus_peers.cc:597] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.677812 24421 tablet_replica.cc:333] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.678777 24421 raft_consensus.cc:2243] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.679773 24421 raft_consensus.cc:2272] T 88f6873460634e2eb3253677a6a6b857 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.715719 25676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 49712us
W20260709 12:21:09.721366 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.722146 24421 tablet_replica.cc:333] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:21:09.727834 25677 consensus_peers.cc:597] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:09.732589 24421 raft_consensus.cc:2243] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.734261 24421 raft_consensus.cc:2272] T b74717dca3554e889066a717cfd1527a P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.737401 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:09.739359 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.749181 24421 tablet_replica.cc:333] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.750325 24421 raft_consensus.cc:2243] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.751209 24421 raft_consensus.cc:2272] T 5a760df354a14284b004635d45d88a3e P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.756246 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.767139 24421 tablet_replica.cc:333] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.769568 24421 raft_consensus.cc:2243] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.771062 24421 raft_consensus.cc:2272] T b907e0d1516b4c78a2a977e29402bff9 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.774211 24421 tablet_replica.cc:333] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.775141 24421 raft_consensus.cc:2243] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.776381 24421 raft_consensus.cc:2272] T e905093d8adb4c6797cd77172a30843c P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.786675 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.791484 24421 tablet_replica.cc:333] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.792582 24421 raft_consensus.cc:2243] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.793922 24421 raft_consensus.cc:2272] T 7fee45cced0b4ce18dea86254386fee0 P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.811175 24421 tablet_replica.cc:333] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.812063 24421 raft_consensus.cc:2243] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.813583 24421 raft_consensus.cc:2272] T b5e8bc4d29af4139b4cb0e20fc57b10a P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.816468 24421 tablet_replica.cc:333] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.817351 24421 raft_consensus.cc:2243] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.818243 24421 raft_consensus.cc:2272] T fb1874a5bbce42ad864b2ba3624a0c1c P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:09.830394 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.846619 24421 tablet_replica.cc:333] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.847610 24421 raft_consensus.cc:2243] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:09.848966 24421 raft_consensus.cc:2272] T 2a18426c9efd45e698b2b2ccb80a49e8 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.852386 24421 tablet_replica.cc:333] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.853318 24421 raft_consensus.cc:2243] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.854017 24421 raft_consensus.cc:2272] T b97aeee11fff4d52bb8252149577f56e P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.869656 24421 tablet_replica.cc:333] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.872278  1160 raft_consensus.cc:493] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:21:09.873042  1160 raft_consensus.cc:515] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } attrs { replace: true } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:09.882731 24421 raft_consensus.cc:2243] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:09.884007 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:21:09.885459 24421 raft_consensus.cc:2272] T 593fa615e9b64ba0974d4a40b5307c3c P c7bf8f20701245bcbaa09cc663025f97 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.888890 24421 tablet_replica.cc:333] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.889756 24421 raft_consensus.cc:2243] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:21:09.897411 25677 leader_election.cc:336] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:09.910212 25677 leader_election.cc:304] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:21:09.913081  1145 raft_consensus.cc:2749] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:09.918288 25751 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:09.927810 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60051e5723b2429582407076a003096c" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:21:09.910847 24421 raft_consensus.cc:2272] T e0c8da68209b48f8868677e30067d396 P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.937963  1160 leader_election.cc:290] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:21:09.965111 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:09.941473 24421 tablet_replica.cc:333] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:09.951722  1145 raft_consensus.cc:493] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 10d0c7349085463595aedaf7c071e742)
I20260709 12:21:09.972592  1145 raft_consensus.cc:515] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:09.975055  1145 leader_election.cc:290] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:09.976686 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe9ac51e0a949a7a43e288f3d2ed652" candidate_uuid: "c7bf8f20701245bcbaa09cc663025f97" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:21:09.983260 26308 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:09.984402 24421 raft_consensus.cc:2243] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:09.985522 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:09.989576 24421 raft_consensus.cc:2272] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:09.993399 24421 tablet_replica.cc:333] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
W20260709 12:21:09.982978 25677 leader_election.cc:336] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:10.002363 25677 leader_election.cc:304] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7bf8f20701245bcbaa09cc663025f97; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
W20260709 12:21:10.003126 25677 raft_consensus.cc:2686] T afe9ac51e0a949a7a43e288f3d2ed652 P c7bf8f20701245bcbaa09cc663025f97: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:21:10.004529 24421 raft_consensus.cc:2243] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.006095 24421 raft_consensus.cc:2272] T 9d711fa725d244128d79dfdfb1e70af6 P c7bf8f20701245bcbaa09cc663025f97 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.009379 24421 tablet_replica.cc:333] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:10.010244 24421 raft_consensus.cc:2243] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.011276 24421 raft_consensus.cc:2272] T 79cdf55909074ccd9e6b49679e14434d P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.014230 24421 tablet_replica.cc:333] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:10.015026 24421 raft_consensus.cc:2243] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.015647 24421 raft_consensus.cc:2272] T 60051e5723b2429582407076a003096c P c7bf8f20701245bcbaa09cc663025f97 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.024770 24421 tablet_replica.cc:333] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:10.026942 24421 raft_consensus.cc:2243] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.027850 24421 raft_consensus.cc:2272] T ab8e959328c44aed94ce17546b6198e2 P c7bf8f20701245bcbaa09cc663025f97 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.032907 24421 tablet_replica.cc:333] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:10.034782 24421 raft_consensus.cc:2243] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.036219 24421 raft_consensus.cc:2272] T 0024aee53c2a47e59a06e7e69bfb9eb2 P c7bf8f20701245bcbaa09cc663025f97 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.039160 24421 tablet_replica.cc:333] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:10.039915 24421 raft_consensus.cc:2243] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.041301 24421 raft_consensus.cc:2272] T 402bc370028b4bc097d5c2fb4342cdfa P c7bf8f20701245bcbaa09cc663025f97 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.044329 24421 tablet_replica.cc:333] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:10.048938 24421 raft_consensus.cc:2243] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.050092 24421 raft_consensus.cc:2272] T 571c5e0767064f3e86479b794294d327 P c7bf8f20701245bcbaa09cc663025f97 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.053293 24421 tablet_replica.cc:333] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97: stopping tablet replica
I20260709 12:21:10.054005 24421 raft_consensus.cc:2243] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.055169 24421 raft_consensus.cc:2272] T 85b6713c7d844fb7a5272cac45763c0a P c7bf8f20701245bcbaa09cc663025f97 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:10.139181 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:10.139683  1134 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.140324  1134 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:10.145521 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:21:10.149463  1134 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
W20260709 12:21:10.150321 25751 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:10.150823 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, 10d0c7349085463595aedaf7c071e742
I20260709 12:21:10.151854  1134 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:10.152503 25752 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:10.264842 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:10.319185 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:10.333745  1152 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.334421  1152 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:10.344784  1134 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.345419  1134 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:10.346720 26309 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:10.347514  1152 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.362725 25752 leader_election.cc:336] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:10.364997  1134 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.365381 26308 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:10.365902 26308 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.366655  1152 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:21:10.382660  1152 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.383365  1152 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:10.388296  1152 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:10.388532 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:21:10.389394 25782 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 22.
W20260709 12:21:10.394816 26309 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:10.397959 25751 leader_election.cc:336] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:10.398734  1134 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.399248  1134 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:10.401286  1134 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.409443 25752 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:10.413159 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:10.398440 25751 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:10.413852 26308 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:10.414088  1134 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:21:10.414322 26308 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.418665  1152 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:10.423205 25751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111) [suppressed 519 similar messages]
W20260709 12:21:10.424535 25751 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:10.424961 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.426659  1134 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:10.456012 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:10.468967 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:10.513929  1152 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.514568  1152 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:10.516904  1152 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.534612 26308 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:10.536537 26309 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:10.537101 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.544301  1152 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:10.579768 25751 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:10.642789  1122 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.643483  1122 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:10.645552  1122 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.654893 25751 leader_election.cc:336] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:10.657721 25752 leader_election.cc:336] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:10.658242 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.659050  1122 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:10.660941  1153 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.661554  1153 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:10.663836  1153 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.677501 26308 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:10.681553 26309 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:10.682111 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.683601  1153 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:10.716372 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:10.761059  1122 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.761636  1122 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:10.778903 25751 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:10.781965  1122 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.796836 25752 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:10.797299 25752 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.798015  1122 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:10.826772  1122 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.827314  1122 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:10.829324  1122 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:10.839713 25751 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:10.840461 25752 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:10.840878 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:10.842715  1157 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:21:10.978835 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:10.997300  1134 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:10.998010  1134 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:11.001124  1134 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:11.009845 25751 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:11.016206 26308 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:11.019467  1134 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.020057  1134 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.021940  1134 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:11.023319 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:21:11.024888  1153 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.025558  1153 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.030529  1153 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:11.026178 26309 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.021445 25752 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.035229 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.035970  1134 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:21:11.054801 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
W20260709 12:21:11.066985 26309 leader_election.cc:336] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.072005 26307 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.073086  1152 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:11.079569 25752 leader_election.cc:336] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.080045 25752 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 10d0c7349085463595aedaf7c071e742, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.080721  1134 raft_consensus.cc:2749] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:21:11.116964 25752 consensus_peers.cc:597] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.149211  1157 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.149871  1157 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:11.150158  1152 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.150908  1152 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.154422  1122 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.154942  1122 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:11.156697  1157 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:11.157361  1157 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.158083  1134 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.156755  1122 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:11.158651  1134 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.160580  1134 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:11.162195 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:21:11.162947 26342 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 15.
I20260709 12:21:11.164573 25750 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
I20260709 12:21:11.164906 25782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348" is_pre_election: true
I20260709 12:21:11.165571 25782 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 14.
W20260709 12:21:11.167760 26309 leader_election.cc:336] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.168560  1134 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:21:11.168951  1134 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.169304  1134 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:21:11.174602  1152 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:11.175568 25752 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.175621 26307 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.177067  1152 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:21:11.177477  1152 raft_consensus.cc:493] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.177881  1152 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:21:11.157861  1157 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:11.180617  1157 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:11.184446 25752 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:11.185024 25752 leader_election.cc:336] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.188161  1134 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.191589 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fee45cced0b4ce18dea86254386fee0" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742"
I20260709 12:21:11.192337 26342 raft_consensus.cc:3060] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:21:11.197525  1134 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:11.205672 26342 raft_consensus.cc:2468] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 16.
W20260709 12:21:11.210775 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.217224 25751 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.217617 25750 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 
W20260709 12:21:11.218410 25751 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.218690  1134 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:11.215838 25752 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:11.219450 25752 consensus_peers.cc:597] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.219774  1164 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.190852  1152 raft_consensus.cc:515] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.220448  1164 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:11.219465  1134 raft_consensus.cc:2804] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:21:11.225678  1164 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:11.225996  1134 raft_consensus.cc:697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Becoming Leader. State: Replica: 955233c139344131a847044f17f8d348, State: Running, Role: LEADER
I20260709 12:21:11.218943 25751 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.227517  1152 leader_election.cc:290] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Requested vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:11.227582  1134 consensus_queue.cc:237] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:11.232213 26309 leader_election.cc:336] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.228410  1122 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:21:11.228714 25751 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.234421 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.235155  1122 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:11.225687 25752 leader_election.cc:336] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:11.245886 24421 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 25734)
W20260709 12:21:11.248234 25751 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.253201 25780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e905093d8adb4c6797cd77172a30843c" candidate_uuid: "10d0c7349085463595aedaf7c071e742" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "955233c139344131a847044f17f8d348"
I20260709 12:21:11.253944 25780 raft_consensus.cc:3060] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:21:11.282694 25528 catalog_manager.cc:5697] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 reported cstate change: term changed from 15 to 16, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 955233c139344131a847044f17f8d348 (127.23.217.67). New cstate: current_term: 16 leader_uuid: "955233c139344131a847044f17f8d348" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:11.286540  1134 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.287139  1134 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:11.289031  1134 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:11.306516  1153 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.307155  1153 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.309473  1153 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:11.320348 25751 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:11.324265 26309 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:11.342247 26308 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.343288 26308 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:11.352108 25752 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.355254 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.339040 25780 raft_consensus.cc:2468] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d0c7349085463595aedaf7c071e742 in term 15.
I20260709 12:21:11.356048  1152 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:21:11.359493  1134 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:21:11.383671 25752 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.384217 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.385053  1122 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:11.402169 25752 consensus_peers.cc:597] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.410825 25752 consensus_peers.cc:597] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.437042 26309 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.438377 26307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:41061, user_credentials={real_user=slave}} blocked reactor thread for 53258.8us
I20260709 12:21:11.439500 26307 leader_election.cc:304] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.441576  1168 raft_consensus.cc:2804] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:21:11.443638 25752 consensus_peers.cc:597] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.443650  1168 raft_consensus.cc:697] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Becoming Leader. State: Replica: 10d0c7349085463595aedaf7c071e742, State: Running, Role: LEADER
I20260709 12:21:11.446039  1168 consensus_queue.cc:237] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:11.448192 26309 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.454723 26309 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.476074 26309 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.479305 25752 consensus_peers.cc:597] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.487594 26309 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.493109 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:11.516831  1153 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.517532  1153 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.520859  1153 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:11.526602 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:11.527623 26308 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.528244 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.529541  1153 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:21:11.539999 25533 catalog_manager.cc:5697] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 reported cstate change: term changed from 14 to 15, leader changed from c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66) to 10d0c7349085463595aedaf7c071e742 (127.23.217.68). New cstate: current_term: 15 leader_uuid: "10d0c7349085463595aedaf7c071e742" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:11.550073 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:11.576577  1122 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.577219  1122 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:11.585467 25751 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.586876  1122 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:11.591691 25752 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:11.592217 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.593705  1122 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:11.613440 25752 consensus_peers.cc:597] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.642999 25752 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.665076 25752 consensus_peers.cc:597] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.679876 25752 consensus_peers.cc:597] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.682447 26338 raft_consensus.cc:1275] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Refusing update from remote peer 955233c139344131a847044f17f8d348: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:21:11.688928  1122 consensus_queue.cc:1048] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:21:11.713337 26309 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.725344 26309 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.727412 25752 consensus_peers.cc:597] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.757234 25752 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.757822 25752 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.766224 25752 consensus_peers.cc:597] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.813884 25752 consensus_peers.cc:597] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.822255 25752 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.848129 26309 consensus_peers.cc:597] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.856560 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:11.860792 26308 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:11.869145 26309 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.904563  1122 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.905097  1122 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:11.912081  1122 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:11.913005 26338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
I20260709 12:21:11.913724 26338 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
W20260709 12:21:11.918329 25752 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:11.924825 25751 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:11.925366 25751 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:11.926175  1134 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:11.966655 25782 raft_consensus.cc:1275] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Refusing update from remote peer 10d0c7349085463595aedaf7c071e742: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:21:11.968281  1168 consensus_queue.cc:1048] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:21:11.970935 26309 consensus_peers.cc:597] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.981189  1134 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:11.981855  1134 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:11.984709  1134 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.028512 25751 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:12.037871 25752 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:12.039525  1153 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:12.040051  1153 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:12.042984  1153 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.052591 26309 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:12.053187 26308 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:12.053719 26308 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.056842  1153 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:21:12.038796 25752 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.086882  1134 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:12.092971 25752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:39947, user_credentials={real_user=slave}} blocked reactor thread for 55127.7us
W20260709 12:21:12.098285 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:12.120939  1134 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:12.121533  1134 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:12.121811 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:12.123602  1134 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.128671 25751 leader_election.cc:336] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:12.133903 25752 leader_election.cc:336] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:12.134346 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.137873  1134 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:21:12.154464 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:12.198626  1170 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:12.199307  1170 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:12.206671 26309 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:12.207746  1170 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.213197 26308 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:12.213752 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.214748  1153 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:12.246403 24421 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 25734)
W20260709 12:21:12.251451 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:12.301133  1170 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:21:12.301436 25751 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:12.301819  1170 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:12.309199 26308 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:12.316461 26309 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:12.317031 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.317843  1170 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:12.318434  1153 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:12.331172 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:12.341712 25751 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:12.392951 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:12.403487 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:12.415737 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:12.485705 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:12.511624 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:12.578436 26308 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:12.579862 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:12.582118  1177 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:12.585291  1177 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:12.596966 25751 leader_election.cc:336] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:12.599314  1177 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.606876 25752 leader_election.cc:336] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:12.608536 25752 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.609925  1177 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:12.624395 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:12.809392  1122 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:12.809906  1122 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:12.811861  1122 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.826517 25752 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:12.829164 25751 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:12.829684 25751 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.831580  1122 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:21:12.849176  1122 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:12.849788  1122 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:12.857324  1122 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.877030 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:12.877396 25751 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:12.925865 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:12.918251 25752 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:12.927052 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:12.933732 25751 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:12.928213  1122 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:12.936753  1134 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:12.937577  1134 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:12.940994  1134 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:12.951872 25752 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:12.957790 25751 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:12.958830 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:12.963208  1180 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:12.985652 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:12.991153 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:13.044245  1153 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.044771  1153 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:13.052834  1153 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:13.065431 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:13.075139 26308 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.075589 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.076300  1153 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:21:13.086795  1134 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.087321  1134 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:13.089193  1134 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:13.093896 25752 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:13.096622 25751 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.097043 25751 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.098165  1134 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:21:13.120059  1134 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.120539  1134 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:13.122426  1134 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:13.127151 25752 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:13.129498 25751 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.129874 25751 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.131772  1180 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:21:13.141727 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:13.204984  1134 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.205549  1134 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:13.207756  1134 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:13.219326 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:13.220137 25752 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:13.220541 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.221199  1134 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:13.246898 24421 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 25734)
I20260709 12:21:13.340250  1180 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.342293  1180 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:13.381618  1177 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.382519  1177 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:13.383399 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:13.384194 25751 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.385020  1177 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:13.370280  1180 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:13.392746 25751 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.422030  1153 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.426440  1153 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:13.433622 25752 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:13.434278 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:13.435683 25752 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:13.436630 26308 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.436933 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.431049  1153 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:13.436717  1180 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:21:13.439018 26309 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:13.439707 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.439886  1177 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:13.442957  1153 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:21:13.466740  1180 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.467576  1180 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:13.470180  1180 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:13.474839 25751 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:13.485527 25752 consensus_peers.cc:597] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:13.500751  1180 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
W20260709 12:21:13.501730 25752 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:13.501623  1180 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:13.502197 25752 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.505039  1177 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:13.508345  1180 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:13.509826 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:21:13.511536 25751 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.512010 26340 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
W20260709 12:21:13.530900 25752 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:13.531585 25752 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:13.538692  1180 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:13.646822 26308 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:13.717381 26309 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.721032 26309 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.777806 25752 consensus_peers.cc:597] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.822681 25752 consensus_peers.cc:597] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.823705 26309 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.846728 26309 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.853875 26309 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.846753 25752 consensus_peers.cc:597] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.863248 25752 consensus_peers.cc:597] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:13.932977 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:13.939592  1170 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:13.940447  1170 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:13.955478 26309 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:13.961638  1170 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:13.988935 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:13.994659 26308 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:13.996163 26308 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:14.004135  1170 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:14.031834 25752 consensus_peers.cc:597] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.125317 26309 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.127631 25752 consensus_peers.cc:597] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:14.130146  1180 raft_consensus.cc:493] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:14.130882  1180 raft_consensus.cc:515] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:14.133286  1180 leader_election.cc:290] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:14.139298 25752 leader_election.cc:336] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:14.141093 25751 leader_election.cc:336] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:14.141564 25751 leader_election.cc:304] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:14.143301  1177 raft_consensus.cc:2749] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:14.145504  1170 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:14.146239  1170 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:14.161448 26308 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:14.162410 26309 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:14.162930 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:14.163507  1170 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:14.164718  1184 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:14.166008 26309 consensus_peers.cc:597] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:14.174638 25752 consensus_peers.cc:597] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.180168 26309 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.186331 26309 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.219436 25752 consensus_peers.cc:597] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.228179 25752 consensus_peers.cc:597] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.247591 24421 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 25734)
W20260709 12:21:14.248311 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:14.273768 25752 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.292829 26309 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.382071 25752 consensus_peers.cc:597] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:14.382737  1180 raft_consensus.cc:493] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:14.383575  1180 raft_consensus.cc:515] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:14.386047  1180 leader_election.cc:290] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:14.394409 25751 leader_election.cc:336] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:14.394977 25752 leader_election.cc:336] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:14.395653 25752 leader_election.cc:304] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:14.397313  1180 raft_consensus.cc:2749] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:21:14.426415 26309 consensus_peers.cc:597] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.436918 25751 consensus_peers.cc:597] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:14.438809 25752 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.452069 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:14.460668 25752 consensus_peers.cc:597] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.572576 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:14.585773 26308 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:14.605409 25752 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.611110 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:14.622939  1180 raft_consensus.cc:493] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:14.623764  1180 raft_consensus.cc:515] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:14.638702 25751 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:14.626308  1180 leader_election.cc:290] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:14.655359 25752 leader_election.cc:336] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:14.656683 25752 leader_election.cc:304] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:14.659444  1153 raft_consensus.cc:493] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:14.660753  1153 raft_consensus.cc:515] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:14.663261  1153 leader_election.cc:290] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:14.666855  1180 raft_consensus.cc:2749] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:21:14.672963 26308 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:14.700783 26309 leader_election.cc:336] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:14.708411 26309 leader_election.cc:304] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:14.710093  1170 raft_consensus.cc:2749] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:14.707275 25752 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.815441 25751 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:14.816342  1180 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:14.817386  1180 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:14.825704 25752 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:14.828778  1180 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:14.831977 25751 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:14.832561 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:14.833827  1180 raft_consensus.cc:2749] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:14.838018 25521 catalog_manager.cc:4755] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e905093d8adb4c6797cd77172a30843c: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:14.858552 25751 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:14.889724 26308 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:14.907394 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:14.914963 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:14.973739 25751 consensus_peers.cc:597] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:14.977403 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:15.014698 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:15.016779 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:15.059011  1185 raft_consensus.cc:493] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:15.059635  1185 raft_consensus.cc:515] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:15.062064  1185 leader_election.cc:290] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:15.070233 25751 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:15.081036 25752 leader_election.cc:336] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:15.081547 25752 leader_election.cc:304] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:15.082775  1185 raft_consensus.cc:2749] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:15.085695 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:15.248152 24421 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 25734)
W20260709 12:21:15.257899 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:15.301743 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:15.370054  1189 raft_consensus.cc:493] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:15.370887  1189 raft_consensus.cc:515] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:15.377964 25751 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:15.382740  1189 leader_election.cc:290] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:15.397343 25752 leader_election.cc:336] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:15.398015 25752 leader_election.cc:304] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:15.400098  1190 raft_consensus.cc:2749] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:15.404142 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:15.406124  1190 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:15.406908  1190 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:15.413372  1190 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:15.415023 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:15.429446 25752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111) [suppressed 629 similar messages]
W20260709 12:21:15.445700 25752 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:15.446369 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:15.447230  1190 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:15.447897  1190 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:15.456066 25751 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:15.458213  1189 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:15.464630 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd362b7abdb43de96e8b4d518fbb0f9" candidate_uuid: "955233c139344131a847044f17f8d348" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "10d0c7349085463595aedaf7c071e742" is_pre_election: true
W20260709 12:21:15.465204 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:15.465267 26340 raft_consensus.cc:2468] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955233c139344131a847044f17f8d348 in term 22.
I20260709 12:21:15.467537  1190 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023)
I20260709 12:21:15.488757  1177 raft_consensus.cc:493] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:15.489365  1177 raft_consensus.cc:515] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
W20260709 12:21:15.495769 25751 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:15.498749 25752 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:15.501356  1177 leader_election.cc:290] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:15.502177 25752 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742, 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:15.503132  1177 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:15.526028 25752 leader_election.cc:336] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:15.526684 25752 leader_election.cc:304] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:15.527511  1177 raft_consensus.cc:2749] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:21:15.533502 25751 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:15.556521 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:15.726389 25752 consensus_peers.cc:597] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:15.745795 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:15.750777  1177 raft_consensus.cc:493] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:15.751614  1177 raft_consensus.cc:515] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:15.753979  1191 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:15.755128  1191 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:15.767493 25752 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:15.768182 26308 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:15.753958  1177 leader_election.cc:290] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:15.770121 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:15.771782 26309 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:15.773389  1191 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:15.768591 25751 leader_election.cc:336] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:15.775125 25751 leader_election.cc:304] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:15.775760  1153 raft_consensus.cc:2749] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:21:15.778798  1177 raft_consensus.cc:2749] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:15.926802 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
W20260709 12:21:15.957587 25751 consensus_peers.cc:597] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:15.958382 26309 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.130054 25752 consensus_peers.cc:597] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:16.143299  1190 raft_consensus.cc:493] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:16.143942  1190 raft_consensus.cc:515] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:16.146219  1190 leader_election.cc:290] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:16.148380 26309 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.152190 25752 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:16.154455 25751 leader_election.cc:336] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:16.154986 25751 leader_election.cc:304] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:16.156342  1190 raft_consensus.cc:2749] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:16.207252 26309 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.226614 26309 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.256376 26309 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.370434 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:16.398171 25752 consensus_peers.cc:597] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:16.413918 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
W20260709 12:21:16.420993 26309 consensus_peers.cc:597] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:16.421392  1177 raft_consensus.cc:493] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:16.422152  1177 raft_consensus.cc:515] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:16.433076  1177 leader_election.cc:290] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:16.437628 25752 consensus_peers.cc:597] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.440176 25751 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:16.442456 25752 leader_election.cc:336] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:16.443058 25752 leader_election.cc:304] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:16.444787  1177 raft_consensus.cc:2749] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:21:16.448421 26308 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:16.467128  1190 raft_consensus.cc:493] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:16.468073  1190 raft_consensus.cc:515] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:16.471937  1190 leader_election.cc:290] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:16.488878 25751 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:16.492251 25752 leader_election.cc:336] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:16.492717 25752 leader_election.cc:304] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:16.519279  1190 raft_consensus.cc:2749] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:21:16.527669 25752 consensus_peers.cc:597] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.529807 25752 consensus_peers.cc:597] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.540560 25752 consensus_peers.cc:597] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.541121 25752 consensus_peers.cc:597] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:16.549795 26308 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:16.559813 25752 consensus_peers.cc:597] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.581367 26307 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.586084 26309 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:16.612854 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:16.614460 24421 tablet_replica.cc:333] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.616081 24421 raft_consensus.cc:2243] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.617305 24421 raft_consensus.cc:2272] T 214a36d5c7d94b8ca6fed2b3e1b9ae13 P 955233c139344131a847044f17f8d348 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:16.635608 26307 consensus_peers.cc:597] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.643126 24421 tablet_replica.cc:333] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.646296 24421 raft_consensus.cc:2243] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.648041 24421 raft_consensus.cc:2272] T ef4fd24cfa184aa8a8a1de8f32296e6a P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.652173 24421 tablet_replica.cc:333] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348: stopping tablet replica
W20260709 12:21:16.669435 26307 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.671056 24421 raft_consensus.cc:2243] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.685583 24421 raft_consensus.cc:2272] T 595303b3057546cbbfda4b84f8df8912 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.693957 24421 tablet_replica.cc:333] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348: stopping tablet replica
W20260709 12:21:16.689075 26309 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.687489 25752 consensus_peers.cc:597] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.711364 26307 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.712155 26307 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.743571 24421 raft_consensus.cc:2243] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.749444 24421 raft_consensus.cc:2272] T 5ac10fecbdc4482e93572bd7ff87319e P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.762018 24421 tablet_replica.cc:333] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.767359 24421 raft_consensus.cc:2243] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:21:16.752662 25752 consensus_peers.cc:597] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.768570 26307 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.771360 25751 consensus_peers.cc:597] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:16.774586 26307 consensus_peers.cc:597] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.775563 26307 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.784188 26309 consensus_peers.cc:597] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:16.791422  1191 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:16.792768  1191 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:16.795207  1191 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:16.792184 26309 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.802333 26308 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:16.815513 26309 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:16.816088 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:16.822052 26307 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.823801  1191 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:21:16.804605 24421 raft_consensus.cc:2272] T 294a99dd73c8449f84538cf8965a6467 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:16.829165 26307 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.830312 26307 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.831539 25752 consensus_peers.cc:597] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:16.832765 24421 tablet_replica.cc:333] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.833840 24421 raft_consensus.cc:2243] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.838357 24421 raft_consensus.cc:2272] T 3a039e6df55747e1b662ecb5d069264b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.843365 24421 tablet_replica.cc:333] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.848150 24421 raft_consensus.cc:2243] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:16.846866 25752 consensus_peers.cc:597] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.849808 26307 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.852612 24421 raft_consensus.cc:2272] T f80a8201dcaf4f2a9290c47595f9cf7b P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.856098 24421 tablet_replica.cc:333] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.857193 24421 raft_consensus.cc:2243] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.858116 24421 raft_consensus.cc:2272] T 8c30f8fbe9ff44588b4847921bab9bd0 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.861809 24421 tablet_replica.cc:333] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.863080 24421 raft_consensus.cc:2243] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.864405 24421 raft_consensus.cc:2272] T b1ee3e27f67449aaab6c88508697a762 P 955233c139344131a847044f17f8d348 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.869889 24421 tablet_replica.cc:333] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.870864 24421 raft_consensus.cc:2243] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.872227 24421 raft_consensus.cc:2272] T 19825207f8744cf7809ab169c0ec8973 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.876238 24421 tablet_replica.cc:333] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.877066 24421 raft_consensus.cc:2243] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.877758 24421 raft_consensus.cc:2272] T f6cc82bb18354fbf8395678374e211b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.880807 24421 tablet_replica.cc:333] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.881464 24421 raft_consensus.cc:2243] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.882728 24421 raft_consensus.cc:2272] T 9eec1cb847cc4587be1133c287c515b2 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.884914 24421 tablet_replica.cc:333] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.885916 24421 raft_consensus.cc:2243] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.887310 24421 raft_consensus.cc:2272] T c5dd00c486ca4b45b83fda5cf38079e8 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.890591 24421 tablet_replica.cc:333] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.891582 24421 raft_consensus.cc:2243] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.892819 24421 raft_consensus.cc:2272] T b4174bd2da7548489ad8aa892dabb8bb P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.895910 24421 tablet_replica.cc:333] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.897070 24421 raft_consensus.cc:2243] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.898834 24421 raft_consensus.cc:2272] T 7701d2ef98204ad0bedc9231fe829f69 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.902146 24421 tablet_replica.cc:333] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.903100 24421 raft_consensus.cc:2243] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.904488 24421 raft_consensus.cc:2272] T 6eb70fbead204d9581ca707c05f2befe P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.907327 24421 tablet_replica.cc:333] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348: stopping tablet replica
W20260709 12:21:16.907946 26308 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:16.909911 26309 consensus_peers.cc:597] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:16.908221 24421 raft_consensus.cc:2243] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.910863 24421 raft_consensus.cc:2272] T eb89b7cd7f4d4662ab9d1e1f55af650c P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.913861 24421 tablet_replica.cc:333] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.914896 24421 raft_consensus.cc:2243] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.916088 24421 raft_consensus.cc:2272] T bfd362b7abdb43de96e8b4d518fbb0f9 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.924913 24421 tablet_replica.cc:333] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.925858 24421 raft_consensus.cc:2243] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.926553 24421 raft_consensus.cc:2272] T 60051e5723b2429582407076a003096c P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:16.926605 26307 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.929811 24421 tablet_replica.cc:333] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.930692 24421 raft_consensus.cc:2243] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.932307 24421 raft_consensus.cc:2272] T ab8e959328c44aed94ce17546b6198e2 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.941412 24421 tablet_replica.cc:333] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.942363 24421 raft_consensus.cc:2243] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.943722 24421 raft_consensus.cc:2272] T b9052f9b88ca4b9ebb24b8764ad345af P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.947038 24421 tablet_replica.cc:333] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.947934 24421 raft_consensus.cc:2243] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.949404 24421 raft_consensus.cc:2272] T bd6ac1825a4e4ab9bb0d7d2867816cbe P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.952901 24421 tablet_replica.cc:333] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.960202 24421 raft_consensus.cc:2243] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.962049 24421 raft_consensus.cc:2272] T 2c9b915f0d524c19b87995309e7ea872 P 955233c139344131a847044f17f8d348 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.962049  1177 raft_consensus.cc:493] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:16.962841  1177 raft_consensus.cc:515] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:16.965893 24421 tablet_replica.cc:333] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.966840 24421 raft_consensus.cc:2243] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.968217 24421 raft_consensus.cc:2272] T 31ccbf41d24e4275a7f921db24b940d4 P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.972443 24421 tablet_replica.cc:333] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.973328 24421 raft_consensus.cc:2243] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.974052 24421 raft_consensus.cc:2272] T 88f6873460634e2eb3253677a6a6b857 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.974871  1177 leader_election.cc:290] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:16.981230 25752 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:16.977304 24421 tablet_replica.cc:333] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.984788 24421 raft_consensus.cc:2243] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.986112 24421 raft_consensus.cc:2272] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.989353 24421 tablet_replica.cc:333] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.990449 24421 raft_consensus.cc:2243] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:16.991091 24421 raft_consensus.cc:2272] T b74717dca3554e889066a717cfd1527a P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:16.994221 24421 tablet_replica.cc:333] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:16.995065 24421 raft_consensus.cc:2243] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:16.996441 24421 raft_consensus.cc:2272] T 5a760df354a14284b004635d45d88a3e P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:16.996599 25751 leader_election.cc:336] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:16.997176 25751 leader_election.cc:304] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:16.997891 25751 raft_consensus.cc:2686] T 9aa65389a0ed4ac1b3ec08bd217d5f22 P 955233c139344131a847044f17f8d348: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:21:16.999698 24421 tablet_replica.cc:333] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.002046 24421 raft_consensus.cc:2243] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:17.003549 24421 raft_consensus.cc:2272] T b907e0d1516b4c78a2a977e29402bff9 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:17.015491 26307 consensus_peers.cc:597] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:17.017282 24421 tablet_replica.cc:333] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.018275 24421 raft_consensus.cc:2243] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.019039 24421 raft_consensus.cc:2272] T e905093d8adb4c6797cd77172a30843c P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.022078 24421 tablet_replica.cc:333] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.022871 24421 raft_consensus.cc:2243] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:21:17.025146 24421 raft_consensus.cc:2272] T 7fee45cced0b4ce18dea86254386fee0 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.028189 24421 tablet_replica.cc:333] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.029099 24421 raft_consensus.cc:2243] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.030005 24421 raft_consensus.cc:2272] T c3c8e03aa6454e118cd3f9f25c785bef P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.033319 24421 tablet_replica.cc:333] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.034296 24421 raft_consensus.cc:2243] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.034922 24421 raft_consensus.cc:2272] T b5e8bc4d29af4139b4cb0e20fc57b10a P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.046132 24421 tablet_replica.cc:333] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.047057 24421 raft_consensus.cc:2243] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.048394 24421 raft_consensus.cc:2272] T 2a18426c9efd45e698b2b2ccb80a49e8 P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.058905 24421 tablet_replica.cc:333] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.059772 24421 raft_consensus.cc:2243] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.060839 24421 raft_consensus.cc:2272] T b97aeee11fff4d52bb8252149577f56e P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.064249 24421 tablet_replica.cc:333] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.065353 24421 raft_consensus.cc:2243] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.066200 24421 raft_consensus.cc:2272] T 8a1f6623a51a47759e59269cfa63be4b P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.068943 24421 tablet_replica.cc:333] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.069667 24421 raft_consensus.cc:2243] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.078351 24421 raft_consensus.cc:2272] T a741c75d1e684375b51f415fd0aa0296 P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.081293 24421 tablet_replica.cc:333] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.082176 24421 raft_consensus.cc:2243] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:17.083549 24421 raft_consensus.cc:2272] T e13025a828f141378027526ae4f0b002 P 955233c139344131a847044f17f8d348 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.089568  1177 raft_consensus.cc:493] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:17.090191  1177 raft_consensus.cc:515] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } }
I20260709 12:21:17.092624  1177 leader_election.cc:290] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:17.101994 24421 tablet_replica.cc:333] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.103042 24421 raft_consensus.cc:2243] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.103756 24421 raft_consensus.cc:2272] T 9d711fa725d244128d79dfdfb1e70af6 P 955233c139344131a847044f17f8d348 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.107409 24421 tablet_replica.cc:333] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.108186 24421 raft_consensus.cc:2243] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.109221 24421 raft_consensus.cc:2272] T 0024aee53c2a47e59a06e7e69bfb9eb2 P 955233c139344131a847044f17f8d348 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:17.111089 25751 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:17.112608 24421 tablet_replica.cc:333] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.113966 24421 raft_consensus.cc:2243] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:17.115301 24421 raft_consensus.cc:2272] T 79cdf55909074ccd9e6b49679e14434d P 955233c139344131a847044f17f8d348 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:17.116948 25752 leader_election.cc:336] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:17.117491 25752 leader_election.cc:304] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 955233c139344131a847044f17f8d348; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:17.118425 24421 tablet_replica.cc:333] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.119040  1177 raft_consensus.cc:2749] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:17.119794 24421 raft_consensus.cc:2243] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.120529 24421 raft_consensus.cc:2272] T e0c8da68209b48f8868677e30067d396 P 955233c139344131a847044f17f8d348 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.123464 24421 tablet_replica.cc:333] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.124234 24421 raft_consensus.cc:2243] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:17.125463 24421 raft_consensus.cc:2272] T 593fa615e9b64ba0974d4a40b5307c3c P 955233c139344131a847044f17f8d348 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.128082 24421 tablet_replica.cc:333] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.128705 24421 raft_consensus.cc:2243] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.129346 24421 raft_consensus.cc:2272] T c2f7c369dcd245eeba8804ccb208c79a P 955233c139344131a847044f17f8d348 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.131997 24421 tablet_replica.cc:333] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.132635 24421 raft_consensus.cc:2243] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.133419 24421 raft_consensus.cc:2272] T cb52f5946670453a8ec9b21d12ececbb P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.137347 24421 tablet_replica.cc:333] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348: stopping tablet replica
I20260709 12:21:17.137908 24421 raft_consensus.cc:2243] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.138608 24421 raft_consensus.cc:2272] T 85b6713c7d844fb7a5272cac45763c0a P 955233c139344131a847044f17f8d348 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:17.147377 26308 consensus_peers.cc:597] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:17.214433 26308 consensus_peers.cc:597] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:17.277496 26308 consensus_peers.cc:597] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:17.291544 26308 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:17.320888 26308 consensus_peers.cc:597] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:17.338788 26308 consensus_peers.cc:597] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:17.543545 26308 consensus_peers.cc:597] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:18.272578 26308 consensus_peers.cc:597] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:18.273073 26308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:41483, user_credentials={real_user=slave}} blocked reactor thread for 667024us
I20260709 12:21:18.285095  1197 raft_consensus.cc:493] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.290400  1201 raft_consensus.cc:493] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.291087  1201 raft_consensus.cc:515] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.291622  1202 raft_consensus.cc:493] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.292196  1202 raft_consensus.cc:515] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.293371  1201 leader_election.cc:290] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.294199  1202 leader_election.cc:290] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.301149  1199 raft_consensus.cc:493] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.301621  1199 raft_consensus.cc:515] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.302228  1204 raft_consensus.cc:493] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.303843  1199 leader_election.cc:290] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.302767  1204 raft_consensus.cc:515] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.304337  1199 raft_consensus.cc:493] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.285872  1197 raft_consensus.cc:515] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.309821  1198 raft_consensus.cc:493] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.310441  1198 raft_consensus.cc:515] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.312765  1198 leader_election.cc:290] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.305315  1199 raft_consensus.cc:515] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.314817  1197 leader_election.cc:290] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.315179  1199 leader_election.cc:290] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.315832  1198 raft_consensus.cc:493] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:18.316216  1201 raft_consensus.cc:493] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
W20260709 12:21:18.331348 26308 consensus_peers.cc:597] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:18.331946 26308 consensus_peers.cc:597] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:18.331665  1201 raft_consensus.cc:515] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.310277  1202 raft_consensus.cc:493] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.334905  1202 raft_consensus.cc:515] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } attrs { replace: true } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.331302  1198 raft_consensus.cc:515] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.341516  1198 leader_election.cc:290] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:18.306551  1204 leader_election.cc:290] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.342693  1201 leader_election.cc:290] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.343904  1202 leader_election.cc:290] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:18.344324 24421 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 25808)
W20260709 12:21:18.345064 26308 consensus_peers.cc:597] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:18.345887 26308 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:18.326241 26307 leader_election.cc:336] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.346660 26307 leader_election.cc:336] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.347405 26307 leader_election.cc:336] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.347560 26308 consensus_peers.cc:597] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:18.348778  1202 raft_consensus.cc:493] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.349417  1202 raft_consensus.cc:515] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.351922  1202 leader_election.cc:290] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.354022  1201 raft_consensus.cc:493] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.354614  1201 raft_consensus.cc:515] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.357401  1201 leader_election.cc:290] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.359055  1201 raft_consensus.cc:493] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:18.359704  1201 raft_consensus.cc:515] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } attrs { replace: true } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:18.361011 26308 leader_election.cc:336] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:18.363169 26308 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:18.363399  1201 leader_election.cc:290] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:18.364717 26307 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.365324 26307 leader_election.cc:336] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.365754 26307 leader_election.cc:336] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
I20260709 12:21:18.366187  1201 raft_consensus.cc:493] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.366910  1201 raft_consensus.cc:515] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.368736  1208 raft_consensus.cc:493] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:18.369915  1208 raft_consensus.cc:515] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.372056  1208 leader_election.cc:290] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
W20260709 12:21:18.373214 26307 leader_election.cc:336] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
I20260709 12:21:18.374004  1201 leader_election.cc:290] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:18.373963 26308 consensus_peers.cc:597] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:18.389675  1201 raft_consensus.cc:493] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.390332  1201 raft_consensus.cc:515] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:18.391328 26309 leader_election.cc:336] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.391821 26309 leader_election.cc:304] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.392652  1201 leader_election.cc:290] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:18.393447 26307 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
I20260709 12:21:18.394208  1208 raft_consensus.cc:2749] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:18.394656 26309 leader_election.cc:336] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:18.394187 26307 leader_election.cc:336] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
I20260709 12:21:18.395139 26309 leader_election.cc:304] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.395949  1208 raft_consensus.cc:2749] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:21:18.396246 26309 leader_election.cc:336] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.395511 26307 leader_election.cc:304] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
W20260709 12:21:18.397675 26307 leader_election.cc:336] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
I20260709 12:21:18.398185  1208 raft_consensus.cc:2749] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:18.399425 26307 leader_election.cc:336] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.403209 26307 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.404639 26307 leader_election.cc:336] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
I20260709 12:21:18.396704 26309 leader_election.cc:304] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:18.405179 26308 leader_election.cc:336] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:18.406240  1208 raft_consensus.cc:2749] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:18.407085 26309 leader_election.cc:336] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.407613 26309 leader_election.cc:304] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.408418  1208 raft_consensus.cc:2749] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:21:18.408713 26309 leader_election.cc:336] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.407101 26308 leader_election.cc:304] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:21:18.409353 26309 leader_election.cc:304] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:18.410554 26307 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
I20260709 12:21:18.411396  1208 raft_consensus.cc:2749] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:21:18.412282  1208 raft_consensus.cc:2749] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:21:18.413149 26308 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:18.413528 26309 leader_election.cc:336] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.413591 26308 leader_election.cc:304] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:21:18.414698  1208 raft_consensus.cc:2749] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:18.415416  1208 raft_consensus.cc:493] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:18.415946  1208 raft_consensus.cc:515] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
W20260709 12:21:18.417122 26308 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:18.417613 26308 leader_election.cc:336] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:18.417949 26308 leader_election.cc:304] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:21:18.418675  1201 raft_consensus.cc:493] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.418843  1197 raft_consensus.cc:2749] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:18.419291  1201 raft_consensus.cc:515] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.421802  1201 leader_election.cc:290] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
I20260709 12:21:18.418183  1208 leader_election.cc:290] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:18.414374 26309 leader_election.cc:304] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.423637  1208 raft_consensus.cc:2749] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:18.424906 26309 leader_election.cc:336] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.425434 26309 leader_election.cc:304] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.426244  1208 raft_consensus.cc:2749] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:18.427258 26307 leader_election.cc:336] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.427558 26309 leader_election.cc:336] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.428208 26309 leader_election.cc:304] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.429003  1208 raft_consensus.cc:2749] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:21:18.430069 26309 leader_election.cc:336] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:18.431013 26309 leader_election.cc:336] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.431437 26309 leader_election.cc:304] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:18.431447 26308 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:18.432417  1208 raft_consensus.cc:2749] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:18.433415 26309 leader_election.cc:336] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.433892 26309 leader_election.cc:304] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.434659  1208 raft_consensus.cc:2749] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:21:18.435611 26309 leader_election.cc:336] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
W20260709 12:21:18.439800 26309 leader_election.cc:336] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.440347 26309 leader_election.cc:304] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.441161  1208 raft_consensus.cc:2749] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:18.444977 26309 leader_election.cc:336] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.445480 26309 leader_election.cc:304] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.446324  1208 raft_consensus.cc:2749] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:21:18.446970 26309 leader_election.cc:336] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.447503 26309 leader_election.cc:304] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.448225  1208 raft_consensus.cc:2749] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:21:18.448380 26309 leader_election.cc:336] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.448908 26309 leader_election.cc:304] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.449612  1208 raft_consensus.cc:2749] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:18.469583  1208 raft_consensus.cc:493] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.470140  1208 raft_consensus.cc:515] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.475449  1208 leader_election.cc:290] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:18.477628 26307 leader_election.cc:336] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.478371 26309 leader_election.cc:336] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:18.478930 26309 leader_election.cc:304] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 955233c139344131a847044f17f8d348, c7bf8f20701245bcbaa09cc663025f97
I20260709 12:21:18.479810  1208 raft_consensus.cc:2749] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:18.564685  1208 raft_consensus.cc:493] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 955233c139344131a847044f17f8d348)
I20260709 12:21:18.565317  1208 raft_consensus.cc:515] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "955233c139344131a847044f17f8d348" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 41061 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:18.567425  1208 leader_election.cc:290] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), 955233c139344131a847044f17f8d348 (127.23.217.67:41061)
W20260709 12:21:18.572175 26307 leader_election.cc:336] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111)
W20260709 12:21:18.573338 26308 leader_election.cc:336] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:18.573709 26308 leader_election.cc:304] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, 955233c139344131a847044f17f8d348
I20260709 12:21:18.574388  1208 raft_consensus.cc:2749] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:21:18.691555 26309 consensus_peers.cc:597] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:18.775417 26308 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:18.907227 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
W20260709 12:21:18.925482 26308 consensus_peers.cc:597] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:19.034668 24421 tablet_server.cc:179] TabletServer@127.23.217.68:0 shutting down...
I20260709 12:21:19.091849 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:19.092900 24421 tablet_replica.cc:333] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.094127 24421 raft_consensus.cc:2243] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.095825 24421 raft_consensus.cc:2272] T e905093d8adb4c6797cd77172a30843c P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.098841 24421 tablet_replica.cc:333] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.099562 24421 raft_consensus.cc:2243] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.100165 24421 raft_consensus.cc:2272] T bfd362b7abdb43de96e8b4d518fbb0f9 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.102548 24421 tablet_replica.cc:333] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.103327 24421 raft_consensus.cc:2243] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.104063 24421 raft_consensus.cc:2272] T 0c2fd18ad19c4394a69e6f9d6aa778e1 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.106645 24421 tablet_replica.cc:333] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.107509 24421 raft_consensus.cc:2243] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.108763 24421 raft_consensus.cc:2272] T f80a8201dcaf4f2a9290c47595f9cf7b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.112445 26309 consensus_peers.cc:597] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.112848 24421 tablet_replica.cc:333] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.113862 24421 raft_consensus.cc:2243] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.114967 24421 raft_consensus.cc:2272] T 8a1f6623a51a47759e59269cfa63be4b P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.117558 24421 tablet_replica.cc:333] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.118551 24421 raft_consensus.cc:2243] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.120011 24421 raft_consensus.cc:2272] T afe9ac51e0a949a7a43e288f3d2ed652 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.123231 24421 tablet_replica.cc:333] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.124269 24421 raft_consensus.cc:2243] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.125205 24421 raft_consensus.cc:2272] T 294a99dd73c8449f84538cf8965a6467 P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.127008 26308 consensus_peers.cc:597] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 -> Peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Couldn't send request to peer 0ae979356d694a09be4b0cd976ea354a. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:21:19.128332 24421 tablet_replica.cc:333] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.129078 24421 raft_consensus.cc:2243] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.129637 24421 raft_consensus.cc:2272] T 595303b3057546cbbfda4b84f8df8912 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.132161 24421 tablet_replica.cc:333] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.132787 24421 raft_consensus.cc:2243] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.133677 24421 raft_consensus.cc:2272] T eb89b7cd7f4d4662ab9d1e1f55af650c P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.135861 24421 tablet_replica.cc:333] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.136466 24421 raft_consensus.cc:2243] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.137313 24421 raft_consensus.cc:2272] T 7fee45cced0b4ce18dea86254386fee0 P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.139849 24421 tablet_replica.cc:333] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.140620 24421 raft_consensus.cc:2243] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.141690 24421 raft_consensus.cc:2272] T c3c8e03aa6454e118cd3f9f25c785bef P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.144556 24421 tablet_replica.cc:333] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.145471 24421 raft_consensus.cc:2243] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.146174 24421 raft_consensus.cc:2272] T 31ccbf41d24e4275a7f921db24b940d4 P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.148293 24421 tablet_replica.cc:333] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.148888 24421 raft_consensus.cc:2243] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.150179 24421 raft_consensus.cc:2272] T 60051e5723b2429582407076a003096c P 10d0c7349085463595aedaf7c071e742 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.152433 24421 tablet_replica.cc:333] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.153062 24421 raft_consensus.cc:2243] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.153676 24421 raft_consensus.cc:2272] T ab8e959328c44aed94ce17546b6198e2 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.155882 24421 tablet_replica.cc:333] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.156669 24421 raft_consensus.cc:2243] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.157687 24421 raft_consensus.cc:2272] T 12eaa1f2655a4cc7bde6bd409ccdf780 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.160135 24421 tablet_replica.cc:333] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.160755 24421 raft_consensus.cc:2243] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.161252 24421 raft_consensus.cc:2272] T b907e0d1516b4c78a2a977e29402bff9 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.164179 24421 tablet_replica.cc:333] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.165001 24421 raft_consensus.cc:2243] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.165802 24421 raft_consensus.cc:2272] T 5a760df354a14284b004635d45d88a3e P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.169189 24421 tablet_replica.cc:333] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
W20260709 12:21:19.169162 26309 consensus_peers.cc:597] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.170431 24421 raft_consensus.cc:2243] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.171566 24421 raft_consensus.cc:2272] T b9052f9b88ca4b9ebb24b8764ad345af P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.174013 24421 tablet_replica.cc:333] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.174774 24421 raft_consensus.cc:2243] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.175326 24421 raft_consensus.cc:2272] T 6158b789bb894b2b9ad0c019dde5476f P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.177714 24421 tablet_replica.cc:333] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.178403 24421 raft_consensus.cc:2243] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.179374 24421 raft_consensus.cc:2272] T 4ff8dbae494348a7be4563e3bd2982b6 P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.181571 24421 tablet_replica.cc:333] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.182210 24421 raft_consensus.cc:2243] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.183241 24421 raft_consensus.cc:2272] T 7e41fb20a5d340ab95f58b3ccc5c680f P 10d0c7349085463595aedaf7c071e742 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.185249 24421 tablet_replica.cc:333] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.185832 24421 raft_consensus.cc:2243] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.186870 24421 raft_consensus.cc:2272] T 78bc05d6a56b407bb7507c25a641dc4e P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.189239 24421 tablet_replica.cc:333] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.189805 24421 raft_consensus.cc:2243] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.190752 24421 raft_consensus.cc:2272] T f6cc82bb18354fbf8395678374e211b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.192986 24421 tablet_replica.cc:333] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.193655 24421 raft_consensus.cc:2243] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.194157 24421 raft_consensus.cc:2272] T e13025a828f141378027526ae4f0b002 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.196292 24421 tablet_replica.cc:333] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.196943 24421 raft_consensus.cc:2243] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.198097 24421 raft_consensus.cc:2272] T a741c75d1e684375b51f415fd0aa0296 P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.197986  1202 raft_consensus.cc:493] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c7bf8f20701245bcbaa09cc663025f97)
I20260709 12:21:19.198952  1202 raft_consensus.cc:515] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ae979356d694a09be4b0cd976ea354a" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 41483 } } peers { permanent_uuid: "c7bf8f20701245bcbaa09cc663025f97" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 39947 } } peers { permanent_uuid: "10d0c7349085463595aedaf7c071e742" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 35023 } attrs { promote: false } }
I20260709 12:21:19.200850 24421 tablet_replica.cc:333] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.201318  1202 leader_election.cc:290] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483), c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947)
I20260709 12:21:19.201946 24421 raft_consensus.cc:2243] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.203038 24421 raft_consensus.cc:2272] T 7701d2ef98204ad0bedc9231fe829f69 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.205967 24421 tablet_replica.cc:333] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.206879 24421 raft_consensus.cc:2243] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.207558 24421 raft_consensus.cc:2272] T 9eec1cb847cc4587be1133c287c515b2 P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.208021 26309 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111)
I20260709 12:21:19.210783 24421 tablet_replica.cc:333] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.211810 24421 raft_consensus.cc:2243] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.212502 24421 raft_consensus.cc:2272] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.211598 26308 leader_election.cc:336] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
I20260709 12:21:19.214180 26308 leader_election.cc:304] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10d0c7349085463595aedaf7c071e742; no voters: 0ae979356d694a09be4b0cd976ea354a, c7bf8f20701245bcbaa09cc663025f97
W20260709 12:21:19.214913 26308 raft_consensus.cc:2686] T 571c5e0767064f3e86479b794294d327 P 10d0c7349085463595aedaf7c071e742: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:21:19.215453 24421 tablet_replica.cc:333] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.216369 24421 raft_consensus.cc:2243] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.217049 24421 raft_consensus.cc:2272] T 402bc370028b4bc097d5c2fb4342cdfa P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.217900 26309 consensus_peers.cc:597] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 -> Peer c7bf8f20701245bcbaa09cc663025f97 (127.23.217.66:39947): Couldn't send request to peer c7bf8f20701245bcbaa09cc663025f97. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:39947: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.219375 26307 consensus_peers.cc:597] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 -> Peer 955233c139344131a847044f17f8d348 (127.23.217.67:41061): Couldn't send request to peer 955233c139344131a847044f17f8d348. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.67:41061: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:19.220026 24421 tablet_replica.cc:333] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.220865 24421 raft_consensus.cc:2243] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.221383 24421 raft_consensus.cc:2272] T b1ee3e27f67449aaab6c88508697a762 P 10d0c7349085463595aedaf7c071e742 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.224149 24421 tablet_replica.cc:333] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.224869 24421 raft_consensus.cc:2243] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.225773 24421 raft_consensus.cc:2272] T cb52f5946670453a8ec9b21d12ececbb P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.228000 24421 tablet_replica.cc:333] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.228608 24421 raft_consensus.cc:2243] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.229059 24421 raft_consensus.cc:2272] T ef4fd24cfa184aa8a8a1de8f32296e6a P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.231137 24421 tablet_replica.cc:333] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.231696 24421 raft_consensus.cc:2243] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.232127 24421 raft_consensus.cc:2272] T 5ac10fecbdc4482e93572bd7ff87319e P 10d0c7349085463595aedaf7c071e742 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.234309 24421 tablet_replica.cc:333] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.234882 24421 raft_consensus.cc:2243] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.235980 24421 raft_consensus.cc:2272] T fb1874a5bbce42ad864b2ba3624a0c1c P 10d0c7349085463595aedaf7c071e742 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.238444 24421 tablet_replica.cc:333] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.239081 24421 raft_consensus.cc:2243] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.239677 24421 raft_consensus.cc:2272] T 79cdf55909074ccd9e6b49679e14434d P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.242458 24421 tablet_replica.cc:333] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.243266 24421 raft_consensus.cc:2243] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.244324 24421 raft_consensus.cc:2272] T 3a039e6df55747e1b662ecb5d069264b P 10d0c7349085463595aedaf7c071e742 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.246424 24421 tablet_replica.cc:333] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.247026 24421 raft_consensus.cc:2243] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.247797 24421 raft_consensus.cc:2272] T e5735f590eff4fc9ae2686d09f8b6810 P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.249747 24421 tablet_replica.cc:333] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.250272 24421 raft_consensus.cc:2243] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.250815 24421 raft_consensus.cc:2272] T 19825207f8744cf7809ab169c0ec8973 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.252812 24421 tablet_replica.cc:333] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.253357 24421 raft_consensus.cc:2243] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.254208 24421 raft_consensus.cc:2272] T 36db04d76d874d328966edd6be42e2c8 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.256233 24421 tablet_replica.cc:333] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.256840 24421 raft_consensus.cc:2243] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.257838 24421 raft_consensus.cc:2272] T b97aeee11fff4d52bb8252149577f56e P 10d0c7349085463595aedaf7c071e742 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.260509 24421 tablet_replica.cc:333] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.261132 24421 raft_consensus.cc:2243] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.262336 24421 raft_consensus.cc:2272] T 2c9b915f0d524c19b87995309e7ea872 P 10d0c7349085463595aedaf7c071e742 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.264846 24421 tablet_replica.cc:333] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.265364 24421 raft_consensus.cc:2243] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.266378 24421 raft_consensus.cc:2272] T b06b96578363435cbde83279e86e5c86 P 10d0c7349085463595aedaf7c071e742 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.268677 24421 tablet_replica.cc:333] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.269194 24421 raft_consensus.cc:2243] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.270057 24421 raft_consensus.cc:2272] T 8c30f8fbe9ff44588b4847921bab9bd0 P 10d0c7349085463595aedaf7c071e742 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.272266 24421 tablet_replica.cc:333] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742: stopping tablet replica
I20260709 12:21:19.272750 24421 raft_consensus.cc:2243] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.273252 24421 raft_consensus.cc:2272] T 593fa615e9b64ba0974d4a40b5307c3c P 10d0c7349085463595aedaf7c071e742 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.297147 26365 debug-util.cc:398] Leaking SignalData structure 0x7b08003b6420 after lost signal to thread 24424
W20260709 12:21:19.298610 26365 debug-util.cc:398] Leaking SignalData structure 0x7b080031c280 after lost signal to thread 25570
W20260709 12:21:19.300253 26365 debug-util.cc:398] Leaking SignalData structure 0x7b08002f12c0 after lost signal to thread 26368
I20260709 12:21:20.640231 24421 tablet_server.cc:196] TabletServer@127.23.217.68:0 shutdown complete.
I20260709 12:21:20.730177 24421 master.cc:562] Master@127.23.217.126:36567 shutting down...
I20260709 12:21:20.766960 25525 maintenance_manager.cc:419] P 3c2fa83ead874145ba497a2c6d2a12d0: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033384
I20260709 12:21:20.799645 25524 maintenance_manager.cc:643] P 3c2fa83ead874145ba497a2c6d2a12d0: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.031s	user 0.024s	sys 0.004s Metrics: {"bytes_written":6848,"delete_count":0,"lbm_write_time_us":869,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":764,"threads_started":1,"update_count":266,"wal-append.queue_time_us":1123}
W20260709 12:21:20.801985 25584 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:21:20.802330 25584 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:21:20.802558 25584 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:21:31.264907 25521 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111) [suppressed 392 similar messages]
W20260709 12:21:31.269215 25521 catalog_manager.cc:4755] TS 0ae979356d694a09be4b0cd976ea354a (127.23.217.65:41483): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet e905093d8adb4c6797cd77172a30843c: Network error: Client connection negotiation failed: client connection to 127.23.217.65:41483: connect: Connection refused (error 111)
W20260709 12:21:47.089493 25520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.68:35023: connect: Connection refused (error 111)
W20260709 12:21:47.093274 25520 catalog_manager.cc:4755] TS 10d0c7349085463595aedaf7c071e742 (127.23.217.68:35023): ChangeConfig:REMOVE_PEER RPC failed for tablet 60051e5723b2429582407076a003096c: Network error: Client connection negotiation failed: client connection to 127.23.217.68:35023: connect: Connection refused (error 111)
I20260709 12:21:47.355548 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:47.357559 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:47.357967 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3c2fa83ead874145ba497a2c6d2a12d0: stopping tablet replica
I20260709 12:21:47.527516 24421 master.cc:584] Master@127.23.217.126:36567 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (282756 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:21:47.646096 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:45747
I20260709 12:21:47.647420 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:47.654320  1221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:47.654417  1220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:47.655146  1223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:47.655789 24421 server_base.cc:1061] running on GCE node
I20260709 12:21:47.659950 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:47.660147 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:47.660356 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599707660338 us; error 0 us; skew 500 ppm
I20260709 12:21:47.660990 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:47.664178 24421 webserver.cc:533] Webserver started at http://127.23.217.126:33327/ using document root <none> and password file <none>
I20260709 12:21:47.664924 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:47.665102 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:47.665342 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:47.666725 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "6485732348384e9795affe8a32c3126e"
format_stamp: "Formatted at 2026-07-09 12:21:47 on dist-test-slave-lg8f"
I20260709 12:21:47.671509 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:21:47.675125  1228 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:47.676052 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:47.676419 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "6485732348384e9795affe8a32c3126e"
format_stamp: "Formatted at 2026-07-09 12:21:47 on dist-test-slave-lg8f"
I20260709 12:21:47.676690 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:47.699118 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:47.700567 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:47.744383 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:45747
I20260709 12:21:47.744563  1279 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:45747 every 8 connection(s)
I20260709 12:21:47.749397  1280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.763010  1280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e: Bootstrap starting.
I20260709 12:21:47.768512  1280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.773468  1280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e: No bootstrap required, opened a new log
I20260709 12:21:47.776036  1280 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485732348384e9795affe8a32c3126e" member_type: VOTER }
I20260709 12:21:47.776540  1280 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.776787  1280 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6485732348384e9795affe8a32c3126e, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.777444  1280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485732348384e9795affe8a32c3126e" member_type: VOTER }
I20260709 12:21:47.777951  1280 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:47.778206  1280 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:47.778604  1280 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.784283  1280 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485732348384e9795affe8a32c3126e" member_type: VOTER }
I20260709 12:21:47.784955  1280 leader_election.cc:304] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6485732348384e9795affe8a32c3126e; no voters: 
I20260709 12:21:47.786222  1280 leader_election.cc:290] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:47.786805  1283 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:47.788491  1283 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 1 LEADER]: Becoming Leader. State: Replica: 6485732348384e9795affe8a32c3126e, State: Running, Role: LEADER
I20260709 12:21:47.789305  1283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485732348384e9795affe8a32c3126e" member_type: VOTER }
I20260709 12:21:47.789474  1280 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:47.798563  1284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6485732348384e9795affe8a32c3126e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485732348384e9795affe8a32c3126e" member_type: VOTER } }
I20260709 12:21:47.799227  1284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:47.798857  1285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6485732348384e9795affe8a32c3126e. Latest consensus state: current_term: 1 leader_uuid: "6485732348384e9795affe8a32c3126e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485732348384e9795affe8a32c3126e" member_type: VOTER } }
I20260709 12:21:47.800096  1285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:47.801929  1293 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:47.808611  1293 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:47.809675 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:47.818233  1293 catalog_manager.cc:1383] Generated new cluster ID: a86c15daecb84ba3a347dafb162e6283
I20260709 12:21:47.818614  1293 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:47.832504  1293 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:47.833830  1293 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:47.856773  1293 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e: Generated new TSK 0
I20260709 12:21:47.857337  1293 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:47.876698 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:47.884173  1301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:47.885881  1302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:47.887686  1304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:47.888417 24421 server_base.cc:1061] running on GCE node
I20260709 12:21:47.891306 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:47.891588 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:47.891774 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599707891755 us; error 0 us; skew 500 ppm
I20260709 12:21:47.892470 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:47.895294 24421 webserver.cc:533] Webserver started at http://127.23.217.65:40581/ using document root <none> and password file <none>
I20260709 12:21:47.895835 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:47.896047 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:47.896337 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:47.897746 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "83f9c1c4921946298df299815121637c"
format_stamp: "Formatted at 2026-07-09 12:21:47 on dist-test-slave-lg8f"
I20260709 12:21:47.902858 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.007s
I20260709 12:21:47.906618  1309 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:47.907542 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:21:47.907867 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "83f9c1c4921946298df299815121637c"
format_stamp: "Formatted at 2026-07-09 12:21:47 on dist-test-slave-lg8f"
I20260709 12:21:47.908139 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:47.932544 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:47.933988 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:47.935773 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:47.938607 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:47.938838 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:47.939095 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:47.939293 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:47.988662 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:37337
I20260709 12:21:47.988776  1372 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:37337 every 8 connection(s)
I20260709 12:21:47.994447 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:48.003465  1377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:48.007323  1378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:48.008839 24421 server_base.cc:1061] running on GCE node
W20260709 12:21:48.009742  1380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:48.010989  1373 heartbeater.cc:344] Connected to a master server at 127.23.217.126:45747
I20260709 12:21:48.011384  1373 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:48.013010 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:48.013255 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:48.013273  1373 heartbeater.cc:507] Master 127.23.217.126:45747 requested a full tablet report, sending...
I20260709 12:21:48.013603 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599708013578 us; error 0 us; skew 500 ppm
I20260709 12:21:48.014424 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:48.016070  1245 ts_manager.cc:194] Registered new tserver with Master: 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:48.017583 24421 webserver.cc:533] Webserver started at http://127.23.217.66:34347/ using document root <none> and password file <none>
I20260709 12:21:48.018200 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:48.018407 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:48.018784 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:48.018687  1245 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50420
I20260709 12:21:48.020644 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "95782952bdee421094cada4ea955015d"
format_stamp: "Formatted at 2026-07-09 12:21:48 on dist-test-slave-lg8f"
I20260709 12:21:48.026068 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:21:48.029755  1385 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.030769 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 12:21:48.031098 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "95782952bdee421094cada4ea955015d"
format_stamp: "Formatted at 2026-07-09 12:21:48 on dist-test-slave-lg8f"
I20260709 12:21:48.031435 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:48.042577 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:48.043954 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:48.045626 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:48.048122 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:48.048359 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.048609 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:48.048799 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.097976 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:33559
I20260709 12:21:48.098111  1448 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:33559 every 8 connection(s)
I20260709 12:21:48.103708 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:48.112021  1452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:48.114955  1453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:48.117249  1455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:48.118100 24421 server_base.cc:1061] running on GCE node
I20260709 12:21:48.118995  1449 heartbeater.cc:344] Connected to a master server at 127.23.217.126:45747
I20260709 12:21:48.119117 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:48.119443 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:48.119505  1449 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:48.119673 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599708119651 us; error 0 us; skew 500 ppm
I20260709 12:21:48.120504 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:48.120731  1449 heartbeater.cc:507] Master 127.23.217.126:45747 requested a full tablet report, sending...
I20260709 12:21:48.123313 24421 webserver.cc:533] Webserver started at http://127.23.217.67:44799/ using document root <none> and password file <none>
I20260709 12:21:48.123409  1245 ts_manager.cc:194] Registered new tserver with Master: 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:48.124059 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:48.124334 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:48.124658 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:48.125324  1245 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50434
I20260709 12:21:48.126315 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "e8246e0558124e98a58932ad7f4676e3"
format_stamp: "Formatted at 2026-07-09 12:21:48 on dist-test-slave-lg8f"
I20260709 12:21:48.131212 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:21:48.135397  1460 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.136255 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:21:48.136587 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "e8246e0558124e98a58932ad7f4676e3"
format_stamp: "Formatted at 2026-07-09 12:21:48 on dist-test-slave-lg8f"
I20260709 12:21:48.136914 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:48.150650 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:48.151880 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:48.153393 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:48.155625 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:48.155900 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.156149 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:48.156324 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.201663 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:43099
I20260709 12:21:48.201850  1522 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:43099 every 8 connection(s)
I20260709 12:21:48.206854 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:48.215193  1526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:48.217255  1527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:48.219223  1530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:48.220628 24421 server_base.cc:1061] running on GCE node
I20260709 12:21:48.221833 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:21:48.221899  1523 heartbeater.cc:344] Connected to a master server at 127.23.217.126:45747
W20260709 12:21:48.222160 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:48.222458 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599708222434 us; error 0 us; skew 500 ppm
I20260709 12:21:48.222458  1523 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:48.223342 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:48.223737  1523 heartbeater.cc:507] Master 127.23.217.126:45747 requested a full tablet report, sending...
I20260709 12:21:48.226420 24421 webserver.cc:533] Webserver started at http://127.23.217.68:40971/ using document root <none> and password file <none>
I20260709 12:21:48.226440  1245 ts_manager.cc:194] Registered new tserver with Master: e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:48.227433 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:48.227746 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:48.228109 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:48.228639  1245 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50450
I20260709 12:21:48.229606 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-3-root/instance:
uuid: "76217635dae3410b96baf380d386cbee"
format_stamp: "Formatted at 2026-07-09 12:21:48 on dist-test-slave-lg8f"
I20260709 12:21:48.234344 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.002s
I20260709 12:21:48.238018  1534 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.238986 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:48.239383 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-3-root
uuid: "76217635dae3410b96baf380d386cbee"
format_stamp: "Formatted at 2026-07-09 12:21:48 on dist-test-slave-lg8f"
I20260709 12:21:48.239657 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:48.253794 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:48.255077 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:48.256613 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:48.259058 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:48.259280 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.259542 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:48.259720 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:48.307977 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.68:37405
I20260709 12:21:48.308132  1597 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.68:37405 every 8 connection(s)
I20260709 12:21:48.328503  1598 heartbeater.cc:344] Connected to a master server at 127.23.217.126:45747
I20260709 12:21:48.329073  1598 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:48.329973  1598 heartbeater.cc:507] Master 127.23.217.126:45747 requested a full tablet report, sending...
I20260709 12:21:48.332041  1245 ts_manager.cc:194] Registered new tserver with Master: 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:48.332545 24421 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.021110001s
I20260709 12:21:48.333586  1245 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50458
I20260709 12:21:48.363790  1245 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50470:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:21:48.619601  1488 tablet_service.cc:1511] Processing CreateTablet for tablet 19086f8d6586412f85d639c45f357757 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:48.621222  1488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19086f8d6586412f85d639c45f357757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.620754  1487 tablet_service.cc:1511] Processing CreateTablet for tablet 8576156f1c0b4a5288eb4f9501386641 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:48.622658  1487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8576156f1c0b4a5288eb4f9501386641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.625108  1486 tablet_service.cc:1511] Processing CreateTablet for tablet e4fe4ebb71004bea9cd778d734ff5dff (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:48.627152  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fe4ebb71004bea9cd778d734ff5dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.632388  1413 tablet_service.cc:1511] Processing CreateTablet for tablet 19086f8d6586412f85d639c45f357757 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:48.633947  1413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19086f8d6586412f85d639c45f357757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.638989  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 2d2ac19c44af472cbb267513f0ac353d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:48.639240  1410 tablet_service.cc:1511] Processing CreateTablet for tablet 4af17dabf7ac45919d736585909bca81 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:48.640666  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af17dabf7ac45919d736585909bca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.640793  1412 tablet_service.cc:1511] Processing CreateTablet for tablet 8576156f1c0b4a5288eb4f9501386641 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:48.643024  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8576156f1c0b4a5288eb4f9501386641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.644644  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 4af17dabf7ac45919d736585909bca81 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:48.646530  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af17dabf7ac45919d736585909bca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.646837  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d2ac19c44af472cbb267513f0ac353d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.680387  1409 tablet_service.cc:1511] Processing CreateTablet for tablet db3f7b195a584dd1896512e1497e4ead (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:48.682724  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3f7b195a584dd1896512e1497e4ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.691818  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 54c81e1068f745e09d8d065e567c4b22 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:48.694281  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c81e1068f745e09d8d065e567c4b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.709487  1337 tablet_service.cc:1511] Processing CreateTablet for tablet 2d2ac19c44af472cbb267513f0ac353d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:48.710695  1338 tablet_service.cc:1511] Processing CreateTablet for tablet 8576156f1c0b4a5288eb4f9501386641 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:48.712248  1336 tablet_service.cc:1511] Processing CreateTablet for tablet e4fe4ebb71004bea9cd778d734ff5dff (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:48.716473  1334 tablet_service.cc:1511] Processing CreateTablet for tablet db3f7b195a584dd1896512e1497e4ead (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:48.714304  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d2ac19c44af472cbb267513f0ac353d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.720291  1335 tablet_service.cc:1511] Processing CreateTablet for tablet 4af17dabf7ac45919d736585909bca81 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:48.727695  1619 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:48.730803  1333 tablet_service.cc:1511] Processing CreateTablet for tablet 54c81e1068f745e09d8d065e567c4b22 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:48.732960  1408 tablet_service.cc:1511] Processing CreateTablet for tablet 3ebdd33fb3c0420e8115507583bd63e3 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:48.734606  1408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ebdd33fb3c0420e8115507583bd63e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.739301  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8576156f1c0b4a5288eb4f9501386641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.740463  1333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c81e1068f745e09d8d065e567c4b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.739786  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 3ebdd33fb3c0420e8115507583bd63e3 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:48.741909  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ebdd33fb3c0420e8115507583bd63e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.743320  1335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af17dabf7ac45919d736585909bca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.750370  1334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3f7b195a584dd1896512e1497e4ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.751267  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fe4ebb71004bea9cd778d734ff5dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.759456  1332 tablet_service.cc:1511] Processing CreateTablet for tablet e5a040bff9d840498d2e8f9bf328b13b (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:48.763958  1332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a040bff9d840498d2e8f9bf328b13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.779935  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 19086f8d6586412f85d639c45f357757 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:48.781713  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19086f8d6586412f85d639c45f357757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.785467  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 2d2ac19c44af472cbb267513f0ac353d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:48.785316  1485 tablet_service.cc:1511] Processing CreateTablet for tablet e5a040bff9d840498d2e8f9bf328b13b (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:48.780435  1619 tablet_bootstrap.cc:654] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.787699  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a040bff9d840498d2e8f9bf328b13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.787853  1560 tablet_service.cc:1511] Processing CreateTablet for tablet db3f7b195a584dd1896512e1497e4ead (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:48.790045  1561 tablet_service.cc:1511] Processing CreateTablet for tablet e4fe4ebb71004bea9cd778d734ff5dff (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:48.792214  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 3ebdd33fb3c0420e8115507583bd63e3 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:48.778625  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 54c81e1068f745e09d8d065e567c4b22 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:48.794597  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d2ac19c44af472cbb267513f0ac353d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.802532  1618 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
W20260709 12:21:48.804301  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:48.807657  1557 tablet_service.cc:1511] Processing CreateTablet for tablet 4e617f860b45428ba84e2c8b7511ae75 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:48.809399  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c81e1068f745e09d8d065e567c4b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.810887  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e617f860b45428ba84e2c8b7511ae75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.813028  1618 tablet_bootstrap.cc:654] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.823458  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fe4ebb71004bea9cd778d734ff5dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.823024  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 4e617f860b45428ba84e2c8b7511ae75 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:48.823976  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 4e617f860b45428ba84e2c8b7511ae75 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:48.824661  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e617f860b45428ba84e2c8b7511ae75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.825397  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e617f860b45428ba84e2c8b7511ae75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.827209  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3f7b195a584dd1896512e1497e4ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.837596  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ebdd33fb3c0420e8115507583bd63e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.860648  1486 tablet_service.cc:1511] Processing CreateTablet for tablet fd76a0ea0cf343509a26d9f566ed01a1 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:48.862293  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd76a0ea0cf343509a26d9f566ed01a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.875010  1413 tablet_service.cc:1511] Processing CreateTablet for tablet e5a040bff9d840498d2e8f9bf328b13b (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:48.866384  1556 tablet_service.cc:1511] Processing CreateTablet for tablet fd76a0ea0cf343509a26d9f566ed01a1 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:48.870599  1338 tablet_service.cc:1511] Processing CreateTablet for tablet fd76a0ea0cf343509a26d9f566ed01a1 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:48.876461  1413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a040bff9d840498d2e8f9bf328b13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.876756  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd76a0ea0cf343509a26d9f566ed01a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.875700  1619 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:48.876564  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd76a0ea0cf343509a26d9f566ed01a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.872247  1412 tablet_service.cc:1511] Processing CreateTablet for tablet 4d9872c5f8bf40259f4247e381e78e10 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:48.879667  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d9872c5f8bf40259f4247e381e78e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.880241  1619 ts_tablet_manager.cc:1403] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.153s	user 0.031s	sys 0.013s
I20260709 12:21:48.890944  1487 tablet_service.cc:1511] Processing CreateTablet for tablet 4d9872c5f8bf40259f4247e381e78e10 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:48.892735  1487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d9872c5f8bf40259f4247e381e78e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.899000  1622 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:48.905969  1618 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:48.906638  1618 ts_tablet_manager.cc:1403] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.105s	user 0.012s	sys 0.024s
I20260709 12:21:48.911489  1619 raft_consensus.cc:359] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:48.912325  1619 raft_consensus.cc:385] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.912631  1619 raft_consensus.cc:740] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.920714  1619 consensus_queue.cc:260] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:48.922840  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 4d9872c5f8bf40259f4247e381e78e10 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:48.923911  1618 raft_consensus.cc:359] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:48.924387  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d9872c5f8bf40259f4247e381e78e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.932796  1618 raft_consensus.cc:385] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.933348  1618 raft_consensus.cc:740] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.934325  1618 consensus_queue.cc:260] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:48.969044  1449 heartbeater.cc:499] Master 127.23.217.126:45747 was elected leader, sending a full tablet report...
I20260709 12:21:48.979784  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 2efe5434a3fc4ba1a2552cc148e23a9a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:48.979949  1409 tablet_service.cc:1511] Processing CreateTablet for tablet c396f4bbb0fe496da0c70c8475f9a05e (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:48.981263  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efe5434a3fc4ba1a2552cc148e23a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.981415  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c396f4bbb0fe496da0c70c8475f9a05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.983371  1623 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:48.989579  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 2efe5434a3fc4ba1a2552cc148e23a9a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:48.992918  1523 heartbeater.cc:499] Master 127.23.217.126:45747 was elected leader, sending a full tablet report...
I20260709 12:21:48.993132  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efe5434a3fc4ba1a2552cc148e23a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:48.993934  1622 tablet_bootstrap.cc:654] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.992969  1625 raft_consensus.cc:493] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.001161  1625 raft_consensus.cc:515] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.017203  1485 tablet_service.cc:1511] Processing CreateTablet for tablet c396f4bbb0fe496da0c70c8475f9a05e (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:49.018793  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c396f4bbb0fe496da0c70c8475f9a05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.019673  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 2efe5434a3fc4ba1a2552cc148e23a9a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:49.021020  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2efe5434a3fc4ba1a2552cc148e23a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.021894  1373 heartbeater.cc:499] Master 127.23.217.126:45747 was elected leader, sending a full tablet report...
I20260709 12:21:49.029978  1618 ts_tablet_manager.cc:1434] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.123s	user 0.006s	sys 0.013s
I20260709 12:21:49.015738  1337 tablet_service.cc:1511] Processing CreateTablet for tablet c396f4bbb0fe496da0c70c8475f9a05e (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:49.035111  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c396f4bbb0fe496da0c70c8475f9a05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.031306  1625 leader_election.cc:290] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:49.039948  1623 tablet_bootstrap.cc:654] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.048655  1618 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:49.098155  1618 tablet_bootstrap.cc:654] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.108053  1562 tablet_service.cc:1511] Processing CreateTablet for tablet cb6c9c94c1f54a0ab4c2e3266ce29f1c (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
W20260709 12:21:49.109103  1450 tablet.cc:2406] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:49.111418  1622 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:49.111985  1622 ts_tablet_manager.cc:1403] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.213s	user 0.022s	sys 0.040s
I20260709 12:21:49.113344  1338 tablet_service.cc:1511] Processing CreateTablet for tablet cb6c9c94c1f54a0ab4c2e3266ce29f1c (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:49.114912  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb6c9c94c1f54a0ab4c2e3266ce29f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.115371  1619 ts_tablet_manager.cc:1434] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.235s	user 0.014s	sys 0.017s
I20260709 12:21:49.116539  1619 tablet_bootstrap.cc:492] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:49.115031  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb6c9c94c1f54a0ab4c2e3266ce29f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.124557  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 1508cd4f1b544868bbb54eb5a6f4a6ff (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:49.125684  1623 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:49.126013  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1508cd4f1b544868bbb54eb5a6f4a6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.126201  1623 ts_tablet_manager.cc:1403] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.143s	user 0.015s	sys 0.005s
I20260709 12:21:49.136354  1622 raft_consensus.cc:359] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.137184  1622 raft_consensus.cc:385] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.137511  1622 raft_consensus.cc:740] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.138235  1622 consensus_queue.cc:260] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.140441  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:49.142733  1486 tablet_service.cc:1511] Processing CreateTablet for tablet 1508cd4f1b544868bbb54eb5a6f4a6ff (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:49.144937  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1508cd4f1b544868bbb54eb5a6f4a6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.144505  1623 raft_consensus.cc:359] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:49.147617  1623 raft_consensus.cc:385] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:49.143872  1463 leader_election.cc:343] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.151309  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 1508cd4f1b544868bbb54eb5a6f4a6ff (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:49.153151  1618 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:49.153771  1618 ts_tablet_manager.cc:1403] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.105s	user 0.006s	sys 0.010s
I20260709 12:21:49.117820  1487 tablet_service.cc:1511] Processing CreateTablet for tablet cb6c9c94c1f54a0ab4c2e3266ce29f1c (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:49.154852  1487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb6c9c94c1f54a0ab4c2e3266ce29f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.151391  1623 raft_consensus.cc:740] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.157259  1618 raft_consensus.cc:359] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.157496  1555 tablet_service.cc:1511] Processing CreateTablet for tablet a6de610052e5443095393415714a09b3 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:49.158061  1618 raft_consensus.cc:385] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.158631  1337 tablet_service.cc:1511] Processing CreateTablet for tablet a6de610052e5443095393415714a09b3 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:49.155498  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1508cd4f1b544868bbb54eb5a6f4a6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.160735  1623 consensus_queue.cc:260] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:49.161550  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec6fa8077604b94a27cbc5c876af6be (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:49.160158  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6de610052e5443095393415714a09b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.163185  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec6fa8077604b94a27cbc5c876af6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.159952  1412 tablet_service.cc:1511] Processing CreateTablet for tablet a6de610052e5443095393415714a09b3 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:49.167598  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec6fa8077604b94a27cbc5c876af6be (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:49.164981  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6de610052e5443095393415714a09b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.167968  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 61b61c6a29814aad84e50fcb39c26e56 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:49.165313  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:49.171022  1461 leader_election.cc:343] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.171664  1619 tablet_bootstrap.cc:654] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.171548  1461 leader_election.cc:304] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:49.168130  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6de610052e5443095393415714a09b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.172428  1625 raft_consensus.cc:2749] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.176326  1622 ts_tablet_manager.cc:1434] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.064s	user 0.010s	sys 0.000s
I20260709 12:21:49.176741  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec6fa8077604b94a27cbc5c876af6be (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:49.176923  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec6fa8077604b94a27cbc5c876af6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.180250  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec6fa8077604b94a27cbc5c876af6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.182746  1618 raft_consensus.cc:740] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.190248  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 61b61c6a29814aad84e50fcb39c26e56 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:49.191706  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61b61c6a29814aad84e50fcb39c26e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.195927  1623 ts_tablet_manager.cc:1434] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.069s	user 0.012s	sys 0.001s
I20260709 12:21:49.196810  1598 heartbeater.cc:499] Master 127.23.217.126:45747 was elected leader, sending a full tablet report...
I20260709 12:21:49.197753  1622 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:49.199209  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61b61c6a29814aad84e50fcb39c26e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.201673  1618 consensus_queue.cc:260] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.220484  1632 raft_consensus.cc:493] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.221195  1632 raft_consensus.cc:515] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.222781  1623 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:49.233145  1618 ts_tablet_manager.cc:1434] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.002s
I20260709 12:21:49.236692  1618 tablet_bootstrap.cc:492] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:49.239255  1624 raft_consensus.cc:493] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.239866  1624 raft_consensus.cc:515] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.236279  1338 tablet_service.cc:1511] Processing CreateTablet for tablet 61b61c6a29814aad84e50fcb39c26e56 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:49.242838  1624 leader_election.cc:290] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:49.247534  1619 tablet_bootstrap.cc:492] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:49.248135  1619 ts_tablet_manager.cc:1403] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.000s
I20260709 12:21:49.249439  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 274048e07f194b339d084f93a0259790 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:49.251431  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274048e07f194b339d084f93a0259790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.242864  1632 leader_election.cc:290] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:49.254942  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61b61c6a29814aad84e50fcb39c26e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.256065  1625 raft_consensus.cc:493] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.256624  1625 raft_consensus.cc:515] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.259768  1625 leader_election.cc:290] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:49.260566  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:49.261250  1348 raft_consensus.cc:2468] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:49.262899  1461 leader_election.cc:304] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:49.263967  1625 raft_consensus.cc:2804] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.264425  1625 raft_consensus.cc:493] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.264822  1625 raft_consensus.cc:3060] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.292259  1623 tablet_bootstrap.cc:654] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.293392  1625 raft_consensus.cc:515] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.297927  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 274048e07f194b339d084f93a0259790 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:49.298346  1413 tablet_service.cc:1511] Processing CreateTablet for tablet 00f190b64b2c4391b5e05cf0854cce5b (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:49.299489  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 00f190b64b2c4391b5e05cf0854cce5b (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:49.312873  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f190b64b2c4391b5e05cf0854cce5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.299875  1619 raft_consensus.cc:359] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.277289  1622 tablet_bootstrap.cc:654] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.301672  1618 tablet_bootstrap.cc:654] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.311385  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274048e07f194b339d084f93a0259790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.317632  1560 tablet_service.cc:1511] Processing CreateTablet for tablet f2db5d455eb64aad9a2ab7d8fbf9eeef (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:49.313722  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:49.313735  1486 tablet_service.cc:1511] Processing CreateTablet for tablet 00f190b64b2c4391b5e05cf0854cce5b (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:49.323383  1619 raft_consensus.cc:385] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.323383  1338 tablet_service.cc:1511] Processing CreateTablet for tablet f2db5d455eb64aad9a2ab7d8fbf9eeef (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:49.325754  1410 tablet_service.cc:1511] Processing CreateTablet for tablet f2db5d455eb64aad9a2ab7d8fbf9eeef (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:49.317958  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 274048e07f194b339d084f93a0259790 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:49.312253  1625 leader_election.cc:290] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:49.332541  1348 raft_consensus.cc:3060] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.333464  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2db5d455eb64aad9a2ab7d8fbf9eeef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.335460  1619 raft_consensus.cc:740] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.337672  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2db5d455eb64aad9a2ab7d8fbf9eeef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.341965  1413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f190b64b2c4391b5e05cf0854cce5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.347550  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:49.354982  1424 raft_consensus.cc:2468] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:49.347596  1619 consensus_queue.cc:260] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.351430  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:49.356482  1423 raft_consensus.cc:3060] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.357872  1619 ts_tablet_manager.cc:1434] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.000s
I20260709 12:21:49.358807  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f190b64b2c4391b5e05cf0854cce5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.362039  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274048e07f194b339d084f93a0259790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.362053  1619 tablet_bootstrap.cc:492] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:49.349972  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 3da69d29cbf04086b598c879439a9e8d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:49.382165  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd6baed42964ae8a9fbab7c6e8601c0 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:49.382541  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd6baed42964ae8a9fbab7c6e8601c0 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:49.374639  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2db5d455eb64aad9a2ab7d8fbf9eeef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.383734  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd6baed42964ae8a9fbab7c6e8601c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.344152  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 3da69d29cbf04086b598c879439a9e8d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:49.390349  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da69d29cbf04086b598c879439a9e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.374514  1423 raft_consensus.cc:2468] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:49.379868  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:49.397035  1463 leader_election.cc:304] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:49.398135  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd6baed42964ae8a9fbab7c6e8601c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.398020  1625 raft_consensus.cc:2804] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.402205  1625 raft_consensus.cc:697] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:49.403496  1625 consensus_queue.cc:237] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.404635  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:49.383713  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da69d29cbf04086b598c879439a9e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.406574  1388 leader_election.cc:304] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:49.407822  1348 raft_consensus.cc:2468] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:49.407949  1624 raft_consensus.cc:2749] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.428300  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 38a329febbdb4a35accb4f17719726dc (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:49.429695  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a329febbdb4a35accb4f17719726dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.432840  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd6baed42964ae8a9fbab7c6e8601c0 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:49.434284  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd6baed42964ae8a9fbab7c6e8601c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.435461  1337 tablet_service.cc:1511] Processing CreateTablet for tablet 3da69d29cbf04086b598c879439a9e8d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:49.443643  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3da69d29cbf04086b598c879439a9e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.448737  1623 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:49.449388  1623 ts_tablet_manager.cc:1403] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.234s	user 0.018s	sys 0.009s
I20260709 12:21:49.453738  1633 raft_consensus.cc:493] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.454723  1633 raft_consensus.cc:515] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:49.460188  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:49.461069  1423 raft_consensus.cc:2393] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:49.465085  1633 leader_election.cc:290] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:49.453665  1623 raft_consensus.cc:359] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.470988  1618 tablet_bootstrap.cc:492] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:49.471599  1618 ts_tablet_manager.cc:1403] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.235s	user 0.031s	sys 0.023s
I20260709 12:21:49.485482  1618 raft_consensus.cc:359] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.486284  1618 raft_consensus.cc:385] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.486603  1618 raft_consensus.cc:740] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.486811  1486 tablet_service.cc:1511] Processing CreateTablet for tablet 38a329febbdb4a35accb4f17719726dc (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:49.487031  1412 tablet_service.cc:1511] Processing CreateTablet for tablet 38a329febbdb4a35accb4f17719726dc (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:49.488140  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a329febbdb4a35accb4f17719726dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.488452  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a329febbdb4a35accb4f17719726dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.493602  1556 tablet_service.cc:1511] Processing CreateTablet for tablet fac0e5cd0b264bb18307cc2498883adf (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:49.494930  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac0e5cd0b264bb18307cc2498883adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.477852  1623 raft_consensus.cc:385] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.509234  1623 raft_consensus.cc:740] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.511818  1619 tablet_bootstrap.cc:654] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.519997  1623 consensus_queue.cc:260] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.569531  1623 ts_tablet_manager.cc:1434] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.120s	user 0.005s	sys 0.005s
I20260709 12:21:49.570726  1623 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:49.517530  1618 consensus_queue.cc:260] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.571486  1626 raft_consensus.cc:493] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.572027  1626 raft_consensus.cc:515] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.572746  1618 ts_tablet_manager.cc:1434] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.101s	user 0.003s	sys 0.007s
I20260709 12:21:49.555442  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 52095deeb4984630a0d2b2ab844fef7d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:49.576066  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52095deeb4984630a0d2b2ab844fef7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.577913  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19086f8d6586412f85d639c45f357757" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:49.522219  1622 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:49.573982  1618 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:49.522162  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:49.522681  1336 tablet_service.cc:1511] Processing CreateTablet for tablet fac0e5cd0b264bb18307cc2498883adf (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:49.596133  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac0e5cd0b264bb18307cc2498883adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.551091  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:49.591089  1243 catalog_manager.cc:5697] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:49.585402  1424 raft_consensus.cc:2468] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:49.579003  1626 leader_election.cc:290] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:49.531062  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 52095deeb4984630a0d2b2ab844fef7d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:49.537313  1409 tablet_service.cc:1511] Processing CreateTablet for tablet fac0e5cd0b264bb18307cc2498883adf (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:49.594110  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 52095deeb4984630a0d2b2ab844fef7d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:49.542706  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:49.596105  1622 ts_tablet_manager.cc:1403] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.399s	user 0.032s	sys 0.031s
I20260709 12:21:49.603273  1463 leader_election.cc:304] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:49.604416  1626 raft_consensus.cc:2804] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:49.604785  1537 leader_election.cc:343] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.604806  1626 raft_consensus.cc:493] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.606788  1626 raft_consensus.cc:3060] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.607802  1485 tablet_service.cc:1511] Processing CreateTablet for tablet a85193c93a364d41b3124bde25907dee (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:49.609243  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85193c93a364d41b3124bde25907dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.610989  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac0e5cd0b264bb18307cc2498883adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.612489  1312 leader_election.cc:304] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:49.613425  1632 raft_consensus.cc:2749] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.616070  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52095deeb4984630a0d2b2ab844fef7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.616379  1626 raft_consensus.cc:515] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:49.617290  1535 leader_election.cc:343] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.617823  1535 leader_election.cc:304] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:49.618516  1626 leader_election.cc:290] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:49.619159  1633 raft_consensus.cc:2749] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.620815  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52095deeb4984630a0d2b2ab844fef7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.620973  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19086f8d6586412f85d639c45f357757" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:49.623555  1423 raft_consensus.cc:3060] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.625891  1413 tablet_service.cc:1511] Processing CreateTablet for tablet a85193c93a364d41b3124bde25907dee (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:49.639689  1555 tablet_service.cc:1511] Processing CreateTablet for tablet a85193c93a364d41b3124bde25907dee (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:49.640307  1338 tablet_service.cc:1511] Processing CreateTablet for tablet f68fc83ac5034733b526df7c6ffef4f8 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:49.641134  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85193c93a364d41b3124bde25907dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.641942  1619 tablet_bootstrap.cc:492] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:49.642419  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68fc83ac5034733b526df7c6ffef4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.642584  1619 ts_tablet_manager.cc:1403] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.281s	user 0.021s	sys 0.018s
I20260709 12:21:49.648833  1623 tablet_bootstrap.cc:654] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.650616  1622 raft_consensus.cc:359] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.657843  1412 tablet_service.cc:1511] Processing CreateTablet for tablet f68fc83ac5034733b526df7c6ffef4f8 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:49.646193  1619 raft_consensus.cc:359] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.659631  1619 raft_consensus.cc:385] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.659994  1619 raft_consensus.cc:740] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.651844  1622 raft_consensus.cc:385] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.660851  1622 raft_consensus.cc:740] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.661170  1619 consensus_queue.cc:260] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.661654  1622 consensus_queue.cc:260] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.664113  1622 ts_tablet_manager.cc:1434] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.001s
I20260709 12:21:49.665107  1622 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:49.665632  1484 tablet_service.cc:1511] Processing CreateTablet for tablet f68fc83ac5034733b526df7c6ffef4f8 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:49.652426  1413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85193c93a364d41b3124bde25907dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.669111  1619 ts_tablet_manager.cc:1434] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:21:49.670219  1619 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:49.674317  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19086f8d6586412f85d639c45f357757" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:49.675139  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68fc83ac5034733b526df7c6ffef4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.676033  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19086f8d6586412f85d639c45f357757" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
W20260709 12:21:49.677728  1461 leader_election.cc:343] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:49.678795  1461 leader_election.cc:343] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.684911  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68fc83ac5034733b526df7c6ffef4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.693536  1618 tablet_bootstrap.cc:654] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.697896  1632 raft_consensus.cc:493] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.698802  1632 raft_consensus.cc:515] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.701037  1632 leader_election.cc:290] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:49.703011  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:49.704427  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:49.705101  1498 raft_consensus.cc:2468] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 0.
I20260709 12:21:49.706876  1619 tablet_bootstrap.cc:654] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:49.707950  1312 leader_election.cc:343] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.708865  1312 leader_election.cc:304] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 95782952bdee421094cada4ea955015d
I20260709 12:21:49.712244  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 4eef9c46f2ac4be7a66d7f63d7beb50a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:49.713759  1632 raft_consensus.cc:2804] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.731860  1632 raft_consensus.cc:493] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.732448  1632 raft_consensus.cc:3060] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.732285  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 4eef9c46f2ac4be7a66d7f63d7beb50a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:49.738731  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eef9c46f2ac4be7a66d7f63d7beb50a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.738860  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eef9c46f2ac4be7a66d7f63d7beb50a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.741067  1483 tablet_service.cc:1511] Processing CreateTablet for tablet d142957b397e4f75b5526996e950d768 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:49.733282  1423 raft_consensus.cc:2468] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:49.744601  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d142957b397e4f75b5526996e950d768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.744344  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 4eef9c46f2ac4be7a66d7f63d7beb50a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:49.745966  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eef9c46f2ac4be7a66d7f63d7beb50a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.753204  1463 leader_election.cc:304] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:49.754122  1626 raft_consensus.cc:2804] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.754621  1626 raft_consensus.cc:697] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:49.755458  1626 consensus_queue.cc:237] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.756917  1556 tablet_service.cc:1511] Processing CreateTablet for tablet d142957b397e4f75b5526996e950d768 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:49.758260  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d142957b397e4f75b5526996e950d768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.761929  1618 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:49.762794  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 59009719f3a54ceebc3c12b1687b9000 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:49.764043  1618 ts_tablet_manager.cc:1403] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.190s	user 0.009s	sys 0.007s
I20260709 12:21:49.764935  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59009719f3a54ceebc3c12b1687b9000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.768450  1331 tablet_service.cc:1511] Processing CreateTablet for tablet d142957b397e4f75b5526996e950d768 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:49.773706  1623 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:49.774969  1623 ts_tablet_manager.cc:1403] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.205s	user 0.022s	sys 0.013s
I20260709 12:21:49.783452  1624 raft_consensus.cc:493] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.784114  1624 raft_consensus.cc:515] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.784252  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 59009719f3a54ceebc3c12b1687b9000 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:49.785857  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59009719f3a54ceebc3c12b1687b9000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.788009  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:49.788286  1624 leader_election.cc:290] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
W20260709 12:21:49.789690  1388 leader_election.cc:343] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.770382  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d142957b397e4f75b5526996e950d768. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.800545  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 59009719f3a54ceebc3c12b1687b9000 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:49.801332  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:49.802181  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59009719f3a54ceebc3c12b1687b9000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:49.803259  1386 leader_election.cc:343] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.803831  1386 leader_election.cc:304] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
W20260709 12:21:49.807080  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:49.803298  1337 tablet_service.cc:1511] Processing CreateTablet for tablet 6beebaa55c4f41b1b2a0bb635654d53a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:49.808845  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beebaa55c4f41b1b2a0bb635654d53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.809904  1410 tablet_service.cc:1511] Processing CreateTablet for tablet 6beebaa55c4f41b1b2a0bb635654d53a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:49.811300  1624 raft_consensus.cc:2749] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.811977  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beebaa55c4f41b1b2a0bb635654d53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.816365  1623 raft_consensus.cc:359] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:49.817289  1623 raft_consensus.cc:385] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.817618  1623 raft_consensus.cc:740] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.820873  1632 raft_consensus.cc:515] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.782130  1618 raft_consensus.cc:359] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.826984  1618 raft_consensus.cc:385] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.827390  1618 raft_consensus.cc:740] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.828334  1618 consensus_queue.cc:260] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.831355  1618 ts_tablet_manager.cc:1434] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.002s
I20260709 12:21:49.832623  1618 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:49.836084  1619 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:49.836740  1619 ts_tablet_manager.cc:1403] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.167s	user 0.019s	sys 0.012s
I20260709 12:21:49.842255  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 6beebaa55c4f41b1b2a0bb635654d53a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:49.843928  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beebaa55c4f41b1b2a0bb635654d53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.862660  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
W20260709 12:21:49.864737  1312 leader_election.cc:343] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.866101  1632 leader_election.cc:290] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:49.846693  1623 consensus_queue.cc:260] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:49.867389  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:49.868151  1498 raft_consensus.cc:3060] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.869336  1623 ts_tablet_manager.cc:1434] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.001s
I20260709 12:21:49.876231  1498 raft_consensus.cc:2468] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:49.877509  1622 tablet_bootstrap.cc:654] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.877691  1312 leader_election.cc:304] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 95782952bdee421094cada4ea955015d
I20260709 12:21:49.846028  1619 raft_consensus.cc:359] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:49.881547  1619 raft_consensus.cc:385] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.882010  1619 raft_consensus.cc:740] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.882323  1623 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:49.883023  1619 consensus_queue.cc:260] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:49.885810  1619 ts_tablet_manager.cc:1434] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:21:49.881264  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 012be77ba30c48409e5b4e286fa90432 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:49.887208  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012be77ba30c48409e5b4e286fa90432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.890710  1632 raft_consensus.cc:2804] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.898949  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 454ec9762c6f480bbc5587abbce5f8f6 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:49.899686  1410 tablet_service.cc:1511] Processing CreateTablet for tablet 012be77ba30c48409e5b4e286fa90432 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:49.900357  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 454ec9762c6f480bbc5587abbce5f8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.900189  1338 tablet_service.cc:1511] Processing CreateTablet for tablet 012be77ba30c48409e5b4e286fa90432 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:49.904111  1619 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:49.904893  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012be77ba30c48409e5b4e286fa90432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.909541  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 012be77ba30c48409e5b4e286fa90432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.910543  1632 raft_consensus.cc:697] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:21:49.912221  1632 consensus_queue.cc:237] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:49.913058  1618 tablet_bootstrap.cc:654] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.918610  1623 tablet_bootstrap.cc:654] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.925630  1337 tablet_service.cc:1511] Processing CreateTablet for tablet 454ec9762c6f480bbc5587abbce5f8f6 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:49.927403  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 454ec9762c6f480bbc5587abbce5f8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.927739  1633 raft_consensus.cc:493] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.934122  1619 tablet_bootstrap.cc:654] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.933873  1633 raft_consensus.cc:515] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:49.936489  1633 leader_election.cc:290] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:49.928983  1626 consensus_queue.cc:1048] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:21:49.937608  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:49.940845  1535 leader_election.cc:343] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.948520  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 13d20d1ea2914a929e506bd8dd2d5e70 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:49.956043  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 13d20d1ea2914a929e506bd8dd2d5e70 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:49.928545  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 454ec9762c6f480bbc5587abbce5f8f6 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:49.963493  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 454ec9762c6f480bbc5587abbce5f8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.968851  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 13d20d1ea2914a929e506bd8dd2d5e70 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:49.970184  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d20d1ea2914a929e506bd8dd2d5e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.972575  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d20d1ea2914a929e506bd8dd2d5e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.974838  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d20d1ea2914a929e506bd8dd2d5e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.982889  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 09e9e93f05e547be9a7e726a1f4a419f (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:49.984467  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e9e93f05e547be9a7e726a1f4a419f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:49.994108  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:49.995072  1498 raft_consensus.cc:2468] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:49.996811  1537 leader_election.cc:304] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:49.998275  1633 raft_consensus.cc:2804] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.998708  1633 raft_consensus.cc:493] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.999094  1633 raft_consensus.cc:3060] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.021127  1626 consensus_queue.cc:1048] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.025023  1243 catalog_manager.cc:5697] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:50.031502  1622 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:50.032205  1622 ts_tablet_manager.cc:1403] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.367s	user 0.040s	sys 0.018s
I20260709 12:21:50.036190  1622 raft_consensus.cc:359] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.037341  1622 raft_consensus.cc:385] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.037853  1622 raft_consensus.cc:740] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.039297  1622 consensus_queue.cc:260] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.044886  1622 ts_tablet_manager.cc:1434] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:21:50.041410  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 09e9e93f05e547be9a7e726a1f4a419f (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:50.068329  1633 raft_consensus.cc:515] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.070358  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e9e93f05e547be9a7e726a1f4a419f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.054000  1622 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:50.073235  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 09e9e93f05e547be9a7e726a1f4a419f (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:50.075071  1556 tablet_service.cc:1511] Processing CreateTablet for tablet d7f1594b2adc4d949964379d8b1e5295 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:50.076550  1632 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.087620  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e9e93f05e547be9a7e726a1f4a419f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.087620  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f1594b2adc4d949964379d8b1e5295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.089810  1632 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.094075  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:21:50.096107  1312 leader_election.cc:343] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.107143  1623 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:50.104424  1632 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:50.118580  1411 tablet_service.cc:1511] Processing CreateTablet for tablet d7f1594b2adc4d949964379d8b1e5295 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:50.120872  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f1594b2adc4d949964379d8b1e5295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.121229  1618 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:50.121846  1618 ts_tablet_manager.cc:1403] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.289s	user 0.016s	sys 0.005s
I20260709 12:21:50.124070  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 086651b7e7954168abc2acd51a3ed43d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:50.127207  1622 tablet_bootstrap.cc:654] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.127955  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086651b7e7954168abc2acd51a3ed43d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.128520  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 086651b7e7954168abc2acd51a3ed43d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:50.130036  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086651b7e7954168abc2acd51a3ed43d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.131744  1619 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:50.132298  1619 ts_tablet_manager.cc:1403] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.228s	user 0.006s	sys 0.009s
I20260709 12:21:50.134370  1633 leader_election.cc:290] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:50.126650  1623 ts_tablet_manager.cc:1403] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.245s	user 0.016s	sys 0.004s
I20260709 12:21:50.135269  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:50.135911  1498 raft_consensus.cc:3060] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.143999  1619 raft_consensus.cc:359] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.144800  1619 raft_consensus.cc:385] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.145115  1619 raft_consensus.cc:740] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.146912  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
W20260709 12:21:50.148324  1535 leader_election.cc:343] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.150077  1498 raft_consensus.cc:2468] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:50.151299  1537 leader_election.cc:304] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:50.151458  1623 raft_consensus.cc:359] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.152208  1623 raft_consensus.cc:385] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.146214  1619 consensus_queue.cc:260] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.152977  1633 raft_consensus.cc:2804] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.154923  1619 ts_tablet_manager.cc:1434] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 12:21:50.156029  1619 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:50.158078  1245 catalog_manager.cc:5697] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 1 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:50.152546  1623 raft_consensus.cc:740] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.138175  1618 raft_consensus.cc:359] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.175071  1618 raft_consensus.cc:385] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.175485  1618 raft_consensus.cc:740] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.176363  1618 consensus_queue.cc:260] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.183175  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 086651b7e7954168abc2acd51a3ed43d (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:50.185125  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086651b7e7954168abc2acd51a3ed43d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.187135  1623 consensus_queue.cc:260] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.188555  1619 tablet_bootstrap.cc:654] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.188441  1410 tablet_service.cc:1511] Processing CreateTablet for tablet c9d4cf02d3404f36b94ffb724837b032 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:50.189955  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d4cf02d3404f36b94ffb724837b032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.190840  1624 raft_consensus.cc:493] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.191354  1624 raft_consensus.cc:515] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.193473  1624 leader_election.cc:290] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:50.195358  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:50.197260  1386 leader_election.cc:343] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.198194  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
W20260709 12:21:50.199661  1386 leader_election.cc:343] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.200095  1386 leader_election.cc:304] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:50.205005  1624 raft_consensus.cc:2749] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.208962  1337 tablet_service.cc:1511] Processing CreateTablet for tablet d7f1594b2adc4d949964379d8b1e5295 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:50.213044  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f1594b2adc4d949964379d8b1e5295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.215920  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:50.216804  1572 raft_consensus.cc:2468] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 0.
I20260709 12:21:50.202157  1633 raft_consensus.cc:697] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:50.218532  1310 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:50.219519  1633 consensus_queue.cc:237] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.220328  1632 raft_consensus.cc:2804] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.221019  1632 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.221529  1632 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.228499  1618 ts_tablet_manager.cc:1434] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.106s	user 0.009s	sys 0.002s
I20260709 12:21:50.229411  1618 tablet_bootstrap.cc:492] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:50.231863  1626 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.232517  1626 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.235328  1626 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:50.236542  1622 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:50.237092  1622 ts_tablet_manager.cc:1403] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.188s	user 0.017s	sys 0.002s
I20260709 12:21:50.240473  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:50.241115  1572 raft_consensus.cc:2468] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:50.242833  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:50.243191  1483 tablet_service.cc:1511] Processing CreateTablet for tablet c9d4cf02d3404f36b94ffb724837b032 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:50.244580  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d4cf02d3404f36b94ffb724837b032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.245776  1461 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:50.250552  1556 tablet_service.cc:1511] Processing CreateTablet for tablet c9d4cf02d3404f36b94ffb724837b032 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:50.251897  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d4cf02d3404f36b94ffb724837b032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.248143  1626 raft_consensus.cc:2804] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.255445  1626 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.255807  1626 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.255604  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 984e201fabf540b6a2d1a83a6d186454 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:50.257578  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984e201fabf540b6a2d1a83a6d186454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.259311  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 984e201fabf540b6a2d1a83a6d186454 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:50.259806  1623 ts_tablet_manager.cc:1434] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.125s	user 0.013s	sys 0.008s
I20260709 12:21:50.260067  1622 raft_consensus.cc:359] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.260843  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984e201fabf540b6a2d1a83a6d186454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.261031  1623 tablet_bootstrap.cc:492] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:50.279358  1625 consensus_queue.cc:1048] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.260835  1622 raft_consensus.cc:385] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.304702  1338 tablet_service.cc:1511] Processing CreateTablet for tablet fd1ad92910454a39a4ebb1ea41c3c376 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:50.306644  1562 tablet_service.cc:1511] Processing CreateTablet for tablet fd1ad92910454a39a4ebb1ea41c3c376 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:50.308784  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1ad92910454a39a4ebb1ea41c3c376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.299530  1622 raft_consensus.cc:740] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.313285  1622 consensus_queue.cc:260] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.318114  1622 ts_tablet_manager.cc:1434] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.002s
I20260709 12:21:50.319587  1622 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:50.255679  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 984e201fabf540b6a2d1a83a6d186454 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 12:21:50.322121  1461 consensus_peers.cc:597] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 -> Peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Couldn't send request to peer 76217635dae3410b96baf380d386cbee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.323630  1618 tablet_bootstrap.cc:654] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.326959  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1ad92910454a39a4ebb1ea41c3c376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.330101  1633 raft_consensus.cc:493] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.330744  1633 raft_consensus.cc:515] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.333290  1633 leader_election.cc:290] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.335707  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d2ac19c44af472cbb267513f0ac353d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:50.336504  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d2ac19c44af472cbb267513f0ac353d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:50.337944  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984e201fabf540b6a2d1a83a6d186454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.269160  1632 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.341475  1245 catalog_manager.cc:5697] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:50.339567  1535 leader_election.cc:343] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.345794  1411 tablet_service.cc:1511] Processing CreateTablet for tablet fd1ad92910454a39a4ebb1ea41c3c376 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:50.347177  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1ad92910454a39a4ebb1ea41c3c376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.336498  1424 raft_consensus.cc:2468] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:50.357327  1537 leader_election.cc:304] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:50.358839  1347 raft_consensus.cc:2393] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8246e0558124e98a58932ad7f4676e3 in current term 1: Already voted for candidate 83f9c1c4921946298df299815121637c in this term.
I20260709 12:21:50.360852  1623 tablet_bootstrap.cc:654] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:50.361702  1312 consensus_peers.cc:597] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.362797  1633 raft_consensus.cc:2804] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.363265  1633 raft_consensus.cc:493] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.363662  1633 raft_consensus.cc:3060] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.364987  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:50.365885  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:50.366781  1572 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.374598  1632 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:50.375867  1572 raft_consensus.cc:2468] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:50.377326  1619 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:50.377470  1310 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:21:50.378027  1619 ts_tablet_manager.cc:1403] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.222s	user 0.038s	sys 0.019s
I20260709 12:21:50.381189  1626 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.382151  1619 raft_consensus.cc:359] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:50.382273  1632 raft_consensus.cc:2804] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.383023  1633 raft_consensus.cc:515] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.383809  1498 raft_consensus.cc:2393] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:50.386471  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d2ac19c44af472cbb267513f0ac353d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:50.386636  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d2ac19c44af472cbb267513f0ac353d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:50.387317  1424 raft_consensus.cc:3060] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:50.388247  1535 leader_election.cc:343] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.388273  1632 raft_consensus.cc:697] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:21:50.389991  1632 consensus_queue.cc:237] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.391032  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:50.391975  1572 raft_consensus.cc:2393] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8246e0558124e98a58932ad7f4676e3 in current term 1: Already voted for candidate 83f9c1c4921946298df299815121637c in this term.
I20260709 12:21:50.395031  1626 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:50.395320  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:50.397058  1461 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:50.398473  1424 raft_consensus.cc:2468] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:50.398509  1626 raft_consensus.cc:2749] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.400029  1537 leader_election.cc:304] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:50.401109  1670 raft_consensus.cc:2804] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.401621  1670 raft_consensus.cc:697] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:50.402554  1670 consensus_queue.cc:237] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.405308  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 629287ce1cfc44a0b85fcc0dcbde5c07 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:50.406778  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629287ce1cfc44a0b85fcc0dcbde5c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.407191  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 629287ce1cfc44a0b85fcc0dcbde5c07 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:50.415546  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629287ce1cfc44a0b85fcc0dcbde5c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.418943  1633 leader_election.cc:290] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.411800  1337 tablet_service.cc:1511] Processing CreateTablet for tablet 629287ce1cfc44a0b85fcc0dcbde5c07 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:50.383097  1619 raft_consensus.cc:385] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.420183  1619 raft_consensus.cc:740] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.413268  1622 tablet_bootstrap.cc:654] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.418771  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 4f57f5ec0d79454aa37f1c7cd9e30a3a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:50.420253  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 4f57f5ec0d79454aa37f1c7cd9e30a3a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:50.421308  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f57f5ec0d79454aa37f1c7cd9e30a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.421743  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f57f5ec0d79454aa37f1c7cd9e30a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.419893  1410 tablet_service.cc:1511] Processing CreateTablet for tablet 4f57f5ec0d79454aa37f1c7cd9e30a3a (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:50.408272  1618 tablet_bootstrap.cc:492] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:50.429663  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f57f5ec0d79454aa37f1c7cd9e30a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.429334  1619 consensus_queue.cc:260] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:50.429364  1618 ts_tablet_manager.cc:1403] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.200s	user 0.014s	sys 0.000s
I20260709 12:21:50.439906  1623 tablet_bootstrap.cc:492] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:50.440451  1623 ts_tablet_manager.cc:1403] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.180s	user 0.014s	sys 0.000s
I20260709 12:21:50.420308  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 629287ce1cfc44a0b85fcc0dcbde5c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.446936  1245 catalog_manager.cc:5697] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 1 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.451781  1619 ts_tablet_manager.cc:1434] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.073s	user 0.010s	sys 0.000s
I20260709 12:21:50.468298  1618 raft_consensus.cc:359] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.468022  1244 catalog_manager.cc:5697] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:50.469488  1619 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:50.469110  1618 raft_consensus.cc:385] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.472781  1618 raft_consensus.cc:740] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.473594  1618 consensus_queue.cc:260] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.481644  1618 ts_tablet_manager.cc:1434] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.048s	user 0.002s	sys 0.006s
I20260709 12:21:50.458505  1623 raft_consensus.cc:359] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.460213  1331 tablet_service.cc:1511] Processing CreateTablet for tablet c312c9473d0e47e0816684df0c14a27f (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:50.484339  1622 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:50.484629  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c312c9473d0e47e0816684df0c14a27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.485111  1622 ts_tablet_manager.cc:1403] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.166s	user 0.014s	sys 0.003s
I20260709 12:21:50.485913  1484 tablet_service.cc:1511] Processing CreateTablet for tablet c312c9473d0e47e0816684df0c14a27f (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:50.486760  1411 tablet_service.cc:1511] Processing CreateTablet for tablet c312c9473d0e47e0816684df0c14a27f (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:50.492177  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce2de400513464ca379c256dc7b0e89 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:50.492638  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c312c9473d0e47e0816684df0c14a27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.493541  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce2de400513464ca379c256dc7b0e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.497033  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c312c9473d0e47e0816684df0c14a27f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.495292  1618 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:50.495582  1623 raft_consensus.cc:385] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.502660  1623 raft_consensus.cc:740] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.506752  1623 consensus_queue.cc:260] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.509601  1623 ts_tablet_manager.cc:1434] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.003s
I20260709 12:21:50.508658  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce2de400513464ca379c256dc7b0e89 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:50.515316  1632 consensus_queue.cc:1048] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.530016  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce2de400513464ca379c256dc7b0e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.515369  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce2de400513464ca379c256dc7b0e89 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:50.530685  1619 tablet_bootstrap.cc:654] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.531322  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce2de400513464ca379c256dc7b0e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.532258  1623 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:50.520478  1562 tablet_service.cc:1511] Processing CreateTablet for tablet c1bc80953a524e6f844c37036c559c68 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:50.539211  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bc80953a524e6f844c37036c559c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.541244  1623 tablet_bootstrap.cc:654] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.547573  1331 tablet_service.cc:1511] Processing CreateTablet for tablet c1bc80953a524e6f844c37036c559c68 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:50.549145  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bc80953a524e6f844c37036c559c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.553936  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 2236e9eab6544f1f9e89679486b7a6cf (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:50.555464  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2236e9eab6544f1f9e89679486b7a6cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.555982  1410 tablet_service.cc:1511] Processing CreateTablet for tablet 2236e9eab6544f1f9e89679486b7a6cf (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:50.557390  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2236e9eab6544f1f9e89679486b7a6cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.520936  1618 tablet_bootstrap.cc:654] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.562422  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 2236e9eab6544f1f9e89679486b7a6cf (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:50.564241  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2236e9eab6544f1f9e89679486b7a6cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.489437  1622 raft_consensus.cc:359] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.565564  1622 raft_consensus.cc:385] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.566005  1622 raft_consensus.cc:740] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.566852  1622 consensus_queue.cc:260] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.521953  1485 tablet_service.cc:1511] Processing CreateTablet for tablet c1bc80953a524e6f844c37036c559c68 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:50.569793  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bc80953a524e6f844c37036c559c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.569775  1337 tablet_service.cc:1511] Processing CreateTablet for tablet a89da9bea35044babd7f5631e0e7afd8 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:50.571640  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89da9bea35044babd7f5631e0e7afd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.573148  1624 raft_consensus.cc:493] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.573647  1624 raft_consensus.cc:515] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:50.576236  1624 leader_election.cc:290] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:50.578158  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:50.579296  1622 ts_tablet_manager.cc:1434] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.094s	user 0.002s	sys 0.006s
I20260709 12:21:50.580299  1632 raft_consensus.cc:493] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.580804  1632 raft_consensus.cc:515] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.581545  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:50.583420  1632 leader_election.cc:290] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.583655  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:50.584235  1622 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:50.584602  1386 leader_election.cc:304] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:50.585049  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:50.585705  1424 raft_consensus.cc:2468] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 0.
I20260709 12:21:50.585968  1624 raft_consensus.cc:2749] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:50.587280  1310 leader_election.cc:343] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.588397  1312 leader_election.cc:304] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:50.589609  1632 raft_consensus.cc:2804] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.589993  1632 raft_consensus.cc:493] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.590406  1632 raft_consensus.cc:3060] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.590186  1411 tablet_service.cc:1511] Processing CreateTablet for tablet a89da9bea35044babd7f5631e0e7afd8 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:50.591658  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89da9bea35044babd7f5631e0e7afd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.595340  1483 tablet_service.cc:1511] Processing CreateTablet for tablet a89da9bea35044babd7f5631e0e7afd8 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:50.600903  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89da9bea35044babd7f5631e0e7afd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.601086  1655 raft_consensus.cc:493] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.601675  1655 raft_consensus.cc:515] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.653972  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:50.654857  1572 raft_consensus.cc:2468] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 0.
I20260709 12:21:50.657475  1409 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa446bfc89d4e82bd94dd6ce5699591 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:50.658888  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa446bfc89d4e82bd94dd6ce5699591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.659466  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 25b2d9d95dad4ef2ae0f868e611b9eab (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:50.660800  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b2d9d95dad4ef2ae0f868e611b9eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.661285  1655 leader_election.cc:290] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.663378  1619 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:50.663712  1623 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:50.664999  1623 ts_tablet_manager.cc:1403] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.008s
I20260709 12:21:50.665465  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa446bfc89d4e82bd94dd6ce5699591 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:50.666744  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa446bfc89d4e82bd94dd6ce5699591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.668469  1623 raft_consensus.cc:359] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.669288  1623 raft_consensus.cc:385] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.650524  1622 tablet_bootstrap.cc:654] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.672102  1632 raft_consensus.cc:515] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.673060  1310 leader_election.cc:304] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:21:50.672600  1618 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:50.691857  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa446bfc89d4e82bd94dd6ce5699591 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:50.669629  1623 raft_consensus.cc:740] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.686983  1618 ts_tablet_manager.cc:1403] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.192s	user 0.014s	sys 0.029s
I20260709 12:21:50.702042  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 25b2d9d95dad4ef2ae0f868e611b9eab (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:50.688474  1632 leader_election.cc:290] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.686894  1619 ts_tablet_manager.cc:1403] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.218s	user 0.018s	sys 0.031s
I20260709 12:21:50.715737  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa446bfc89d4e82bd94dd6ce5699591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.714643  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:50.725814  1337 tablet_service.cc:1511] Processing CreateTablet for tablet 25b2d9d95dad4ef2ae0f868e611b9eab (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:50.727155  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b2d9d95dad4ef2ae0f868e611b9eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.727427  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5431f408774554975e15a5998e2062 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:50.728729  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5431f408774554975e15a5998e2062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.731791  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5431f408774554975e15a5998e2062 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:50.735352  1413 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1b0961fa034e77877efe85062b2307 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:50.736235  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 6b5431f408774554975e15a5998e2062 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:50.736781  1413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1b0961fa034e77877efe85062b2307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.737689  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5431f408774554975e15a5998e2062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.738418  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 2b11fecd06b84bfba7a373d360972a33 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:50.739461  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1b0961fa034e77877efe85062b2307 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:50.740725  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1b0961fa034e77877efe85062b2307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.741328  1338 tablet_service.cc:1511] Processing CreateTablet for tablet 2b11fecd06b84bfba7a373d360972a33 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:50.742555  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b11fecd06b84bfba7a373d360972a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.736235  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1b0961fa034e77877efe85062b2307 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:50.743964  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1b0961fa034e77877efe85062b2307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.720517  1632 raft_consensus.cc:2804] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.745203  1632 raft_consensus.cc:493] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.745627  1632 raft_consensus.cc:3060] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.756657  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b2d9d95dad4ef2ae0f868e611b9eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.759176  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b5431f408774554975e15a5998e2062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.762449  1670 consensus_queue.cc:1048] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.764043  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b11fecd06b84bfba7a373d360972a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.769312  1618 raft_consensus.cc:359] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:50.770154  1618 raft_consensus.cc:385] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.770478  1618 raft_consensus.cc:740] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.771502  1618 consensus_queue.cc:260] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:50.773934  1618 ts_tablet_manager.cc:1434] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.064s	user 0.005s	sys 0.001s
I20260709 12:21:50.781190  1622 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:50.781869  1622 ts_tablet_manager.cc:1403] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.198s	user 0.016s	sys 0.011s
I20260709 12:21:50.718884  1623 consensus_queue.cc:260] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.783143  1633 raft_consensus.cc:493] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.783689  1633 raft_consensus.cc:515] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.784353  1677 raft_consensus.cc:493] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.784998  1677 raft_consensus.cc:515] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.785787  1483 tablet_service.cc:1511] Processing CreateTablet for tablet c9ff4e1da1794b068ed21d8067b023b0 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:50.786638  1572 raft_consensus.cc:3060] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.787158  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ff4e1da1794b068ed21d8067b023b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.787518  1677 leader_election.cc:290] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:50.789422  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:50.790151  1424 raft_consensus.cc:2468] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:50.791040  1619 raft_consensus.cc:359] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.791878  1619 raft_consensus.cc:385] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.791813  1463 leader_election.cc:304] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
W20260709 12:21:50.715063  1312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 60827.8us
I20260709 12:21:50.791836  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:50.793514  1347 raft_consensus.cc:2393] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76217635dae3410b96baf380d386cbee in current term 1: Already voted for candidate 83f9c1c4921946298df299815121637c in this term.
I20260709 12:21:50.794095  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:50.794652  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:50.794816  1573 raft_consensus.cc:2468] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:50.795394  1423 raft_consensus.cc:3060] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.795553  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:50.797752  1537 leader_election.cc:304] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:50.798528  1633 leader_election.cc:290] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.798893  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:50.798681  1626 raft_consensus.cc:2804] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.800597  1626 raft_consensus.cc:493] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.800987  1626 raft_consensus.cc:3060] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.804873  1572 raft_consensus.cc:2468] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:50.805873  1633 raft_consensus.cc:2749] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.808043  1562 tablet_service.cc:1511] Processing CreateTablet for tablet c9ff4e1da1794b068ed21d8067b023b0 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:50.809644  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ff4e1da1794b068ed21d8067b023b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.809988  1423 raft_consensus.cc:2468] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:50.815173  1618 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:50.816068  1410 tablet_service.cc:1511] Processing CreateTablet for tablet 2b11fecd06b84bfba7a373d360972a33 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:50.836437  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b11fecd06b84bfba7a373d360972a33. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:50.816990  1535 consensus_peers.cc:597] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:50.840788  1618 tablet_bootstrap.cc:654] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.819685  1632 raft_consensus.cc:515] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.792425  1619 raft_consensus.cc:740] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.842399  1619 consensus_queue.cc:260] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.843499  1632 leader_election.cc:290] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.843602  1424 raft_consensus.cc:2468] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 0.
W20260709 12:21:50.819267  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:50.846048  1619 ts_tablet_manager.cc:1434] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.127s	user 0.007s	sys 0.001s
I20260709 12:21:50.847247  1619 tablet_bootstrap.cc:492] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:50.847425  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:50.848101  1573 raft_consensus.cc:3060] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.849378  1623 ts_tablet_manager.cc:1434] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.184s	user 0.011s	sys 0.001s
I20260709 12:21:50.850919  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 4742a96787194d49a2f191cdd0ef8642 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:50.851352  1310 leader_election.cc:304] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:21:50.852260  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4742a96787194d49a2f191cdd0ef8642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.860440  1623 tablet_bootstrap.cc:492] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:50.823356  1412 tablet_service.cc:1511] Processing CreateTablet for tablet 4742a96787194d49a2f191cdd0ef8642 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:50.861579  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4742a96787194d49a2f191cdd0ef8642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.826848  1338 tablet_service.cc:1511] Processing CreateTablet for tablet c9ff4e1da1794b068ed21d8067b023b0 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:50.825932  1486 tablet_service.cc:1511] Processing CreateTablet for tablet 4742a96787194d49a2f191cdd0ef8642 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:50.871479  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4742a96787194d49a2f191cdd0ef8642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.872037  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 0e5f7e35d647437a99929e665c7052eb (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:50.873355  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e5f7e35d647437a99929e665c7052eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.831254  1334 tablet_service.cc:1511] Processing CreateTablet for tablet 0e5f7e35d647437a99929e665c7052eb (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:50.875005  1334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e5f7e35d647437a99929e665c7052eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.875048  1685 consensus_queue.cc:1048] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.822576  1626 raft_consensus.cc:515] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.878610  1632 raft_consensus.cc:2804] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.879446  1632 raft_consensus.cc:697] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:21:50.880414  1618 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:50.881012  1618 ts_tablet_manager.cc:1403] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.000s
I20260709 12:21:50.880609  1632 consensus_queue.cc:237] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.881997  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:50.882512  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:50.883260  1423 raft_consensus.cc:3060] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.883056  1622 raft_consensus.cc:359] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.883863  1622 raft_consensus.cc:385] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.884213  1622 raft_consensus.cc:740] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.884341  1626 leader_election.cc:290] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:50.884240  1619 tablet_bootstrap.cc:654] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.884347  1618 raft_consensus.cc:359] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.885448  1618 raft_consensus.cc:385] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.885243  1622 consensus_queue.cc:260] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.885778  1618 raft_consensus.cc:740] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.886873  1618 consensus_queue.cc:260] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:50.889708  1618 ts_tablet_manager.cc:1434] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:21:50.890911  1618 tablet_bootstrap.cc:492] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:50.892443  1423 raft_consensus.cc:2468] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:50.893059  1633 consensus_queue.cc:1048] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:50.917850  1463 leader_election.cc:304] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:50.919171  1677 raft_consensus.cc:2804] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.919716  1677 raft_consensus.cc:697] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:50.920410  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 58e96847ffff43ca9df16b0fb7a718a9 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:50.920615  1677 consensus_queue.cc:237] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.922151  1685 raft_consensus.cc:493] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.922750  1685 raft_consensus.cc:515] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:50.893340  1622 ts_tablet_manager.cc:1434] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.111s	user 0.003s	sys 0.006s
I20260709 12:21:50.925055  1685 leader_election.cc:290] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:50.925706  1622 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:50.894375  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ff4e1da1794b068ed21d8067b023b0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:50.895725  1312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 60470.4us
I20260709 12:21:50.928874  1573 raft_consensus.cc:2468] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:50.932971  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d2ac19c44af472cbb267513f0ac353d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:50.933884  1422 raft_consensus.cc:2393] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
I20260709 12:21:50.934299  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 58e96847ffff43ca9df16b0fb7a718a9 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:50.935654  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e96847ffff43ca9df16b0fb7a718a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.935840  1623 tablet_bootstrap.cc:654] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.902729  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 0e5f7e35d647437a99929e665c7052eb (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:50.903136  1619 tablet_bootstrap.cc:492] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:50.938715  1619 ts_tablet_manager.cc:1403] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.000s
I20260709 12:21:50.903879  1618 tablet_bootstrap.cc:654] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.882717  1572 raft_consensus.cc:3060] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.939019  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e5f7e35d647437a99929e665c7052eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.940299  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e96847ffff43ca9df16b0fb7a718a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.941884  1310 leader_election.cc:304] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:21:50.944520  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:50.945139  1424 raft_consensus.cc:3060] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.918053  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 58e96847ffff43ca9df16b0fb7a718a9 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:50.947597  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58e96847ffff43ca9df16b0fb7a718a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:50.952980  1347 raft_consensus.cc:3060] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.958762  1685 raft_consensus.cc:2804] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.959640  1422 raft_consensus.cc:3060] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.960318  1619 raft_consensus.cc:359] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.961047  1619 raft_consensus.cc:385] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.961339  1619 raft_consensus.cc:740] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.962204  1619 consensus_queue.cc:260] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:50.963879  1622 tablet_bootstrap.cc:654] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.965198  1619 ts_tablet_manager.cc:1434] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.003s
I20260709 12:21:50.962874  1685 raft_consensus.cc:697] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:21:50.975672  1619 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.003783  1624 raft_consensus.cc:493] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.011479  1483 tablet_service.cc:1511] Processing CreateTablet for tablet 3cda1629f4344ad8b1ebed68a83d0418 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:51.015655  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cda1629f4344ad8b1ebed68a83d0418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:51.020344  1243 catalog_manager.cc:5697] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 1 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.005252  1685 consensus_queue.cc:237] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.035372  1572 raft_consensus.cc:2468] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:51.004653  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d2ac19c44af472cbb267513f0ac353d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.096906  1623 tablet_bootstrap.cc:492] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.101183  1623 ts_tablet_manager.cc:1403] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.241s	user 0.012s	sys 0.020s
I20260709 12:21:51.092160  1626 raft_consensus.cc:493] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.065743  1624 raft_consensus.cc:515] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.112694  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 3cda1629f4344ad8b1ebed68a83d0418 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20260709 12:21:51.113824  1671 log.cc:927] Time spent T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:21:51.114145  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cda1629f4344ad8b1ebed68a83d0418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:51.115492  1624 leader_election.cc:290] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:51.118894  1619 tablet_bootstrap.cc:654] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.125586  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 3cda1629f4344ad8b1ebed68a83d0418 (DEFAULT_TABLE table=test-workload [id=3b4b974b9e8145f684183c53a803c28c]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:51.127220  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cda1629f4344ad8b1ebed68a83d0418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:51.118877  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:51.106101  1623 raft_consensus.cc:359] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.143715  1623 raft_consensus.cc:385] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.144116  1623 raft_consensus.cc:740] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.112222  1626 raft_consensus.cc:515] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.121722  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.150624  1618 tablet_bootstrap.cc:492] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.151162  1618 ts_tablet_manager.cc:1403] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.261s	user 0.036s	sys 0.053s
W20260709 12:21:51.158094  1388 leader_election.cc:343] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.151898  1623 consensus_queue.cc:260] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.159253  1424 raft_consensus.cc:2468] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:51.160395  1310 leader_election.cc:304] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d
I20260709 12:21:51.161499  1626 leader_election.cc:290] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:51.164108  1686 consensus_queue.cc:1048] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.165071  1618 raft_consensus.cc:359] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.165956  1618 raft_consensus.cc:385] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.166306  1618 raft_consensus.cc:740] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.168007  1618 consensus_queue.cc:260] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.170120  1623 ts_tablet_manager.cc:1434] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.067s	user 0.000s	sys 0.008s
I20260709 12:21:51.166174  1685 raft_consensus.cc:2749] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.179050  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
W20260709 12:21:51.204712  1463 leader_election.cc:343] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.179395  1670 consensus_queue.cc:1048] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:51.172701  1386 leader_election.cc:343] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.206671  1386 leader_election.cc:304] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:51.187460  1623 tablet_bootstrap.cc:492] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.210099  1702 raft_consensus.cc:2749] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.188939  1618 ts_tablet_manager.cc:1434] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:21:51.212383  1618 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.217909  1623 tablet_bootstrap.cc:654] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.231629  1618 tablet_bootstrap.cc:654] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:51.223659  1310 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 62060.2us
I20260709 12:21:51.242257  1244 catalog_manager.cc:5697] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.243613  1619 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.244388  1619 ts_tablet_manager.cc:1403] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.269s	user 0.027s	sys 0.020s
I20260709 12:21:51.248354  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.251098  1622 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:51.251793  1622 ts_tablet_manager.cc:1403] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.326s	user 0.029s	sys 0.014s
W20260709 12:21:51.253007  1461 leader_election.cc:343] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.253599  1461 leader_election.cc:304] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:51.254201  1619 raft_consensus.cc:359] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.255045  1619 raft_consensus.cc:385] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.255398  1619 raft_consensus.cc:740] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.255548  1626 raft_consensus.cc:2749] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.255517  1622 raft_consensus.cc:359] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.256822  1622 raft_consensus.cc:385] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.257306  1622 raft_consensus.cc:740] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.259558  1623 tablet_bootstrap.cc:492] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.260092  1623 ts_tablet_manager.cc:1403] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.004s
I20260709 12:21:51.264660  1619 consensus_queue.cc:260] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.271384  1618 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.271931  1618 ts_tablet_manager.cc:1403] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.005s
I20260709 12:21:51.275691  1619 ts_tablet_manager.cc:1434] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:21:51.275454  1618 raft_consensus.cc:359] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.276219  1618 raft_consensus.cc:385] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.276714  1619 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.276927  1618 raft_consensus.cc:740] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.277740  1618 consensus_queue.cc:260] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.279917  1623 raft_consensus.cc:359] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.280594  1623 raft_consensus.cc:385] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.280853  1623 raft_consensus.cc:740] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.281498  1623 consensus_queue.cc:260] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.283492  1623 ts_tablet_manager.cc:1434] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:21:51.285638  1618 ts_tablet_manager.cc:1434] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:21:51.286700  1618 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.287735  1685 consensus_queue.cc:1048] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.291687  1619 tablet_bootstrap.cc:654] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.292934  1626 raft_consensus.cc:493] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.294467  1626 raft_consensus.cc:515] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.295159  1245 catalog_manager.cc:5697] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 1 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.296650  1626 leader_election.cc:290] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.299773  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd76a0ea0cf343509a26d9f566ed01a1" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.301996  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd76a0ea0cf343509a26d9f566ed01a1" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:51.302868  1461 leader_election.cc:343] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:51.303737  1461 leader_election.cc:343] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.304265  1461 leader_election.cc:304] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:51.305490  1677 raft_consensus.cc:493] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.305703  1626 raft_consensus.cc:2749] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.306337  1677 raft_consensus.cc:515] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.293247  1618 tablet_bootstrap.cc:654] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.293738  1623 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.311633  1677 leader_election.cc:290] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:51.311921  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c396f4bbb0fe496da0c70c8475f9a05e" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
W20260709 12:21:51.313653  1463 leader_election.cc:343] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.316395  1623 tablet_bootstrap.cc:654] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.319798  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c396f4bbb0fe496da0c70c8475f9a05e" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:51.322749  1461 leader_election.cc:343] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.323408  1461 leader_election.cc:304] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:51.324416  1677 raft_consensus.cc:2749] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.326121  1572 raft_consensus.cc:3060] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.328105  1708 consensus_queue.cc:1048] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.328097  1623 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.338439  1623 ts_tablet_manager.cc:1403] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.000s
I20260709 12:21:51.258288  1622 consensus_queue.cc:260] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.348951  1623 raft_consensus.cc:359] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.349711  1623 raft_consensus.cc:385] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.349994  1623 raft_consensus.cc:740] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.357622  1623 consensus_queue.cc:260] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.367574  1623 ts_tablet_manager.cc:1434] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.002s
I20260709 12:21:51.368494  1623 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.374214  1623 tablet_bootstrap.cc:654] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.376897  1618 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.377684  1618 ts_tablet_manager.cc:1403] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.003s
I20260709 12:21:51.381172  1619 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.382179  1619 ts_tablet_manager.cc:1403] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.106s	user 0.030s	sys 0.020s
I20260709 12:21:51.383069  1618 raft_consensus.cc:359] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.383828  1618 raft_consensus.cc:385] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.384135  1618 raft_consensus.cc:740] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.385248  1618 consensus_queue.cc:260] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.386795  1677 raft_consensus.cc:493] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.387251  1677 raft_consensus.cc:515] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.384760  1619 raft_consensus.cc:359] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.388371  1619 raft_consensus.cc:385] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.388648  1619 raft_consensus.cc:740] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.389149  1618 ts_tablet_manager.cc:1434] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:21:51.389263  1677 leader_election.cc:290] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.389469  1619 consensus_queue.cc:260] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.391669  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.393625  1619 ts_tablet_manager.cc:1434] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
W20260709 12:21:51.393934  1461 leader_election.cc:343] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.393987  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.394616  1619 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.401266  1619 tablet_bootstrap.cc:654] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:51.401945  1461 leader_election.cc:343] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.402660  1461 leader_election.cc:304] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:51.403489  1677 raft_consensus.cc:2749] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.408718  1670 raft_consensus.cc:493] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.409292  1670 raft_consensus.cc:515] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.411501  1670 leader_election.cc:290] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:51.415593  1619 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.415462  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:51.416299  1619 ts_tablet_manager.cc:1403] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260709 12:21:51.416061  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.415495  1623 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.417222  1623 ts_tablet_manager.cc:1403] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.002s
W20260709 12:21:51.417922  1535 leader_election.cc:343] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:51.419309  1537 leader_election.cc:343] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.419044  1619 raft_consensus.cc:359] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.419881  1537 leader_election.cc:304] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:51.420095  1619 raft_consensus.cc:385] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.420020  1623 raft_consensus.cc:359] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.420610  1619 raft_consensus.cc:740] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.420836  1623 raft_consensus.cc:385] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.421116  1623 raft_consensus.cc:740] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.345674  1622 ts_tablet_manager.cc:1434] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.093s	user 0.009s	sys 0.000s
I20260709 12:21:51.421285  1670 raft_consensus.cc:2749] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.424743  1622 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:51.426775  1677 consensus_queue.cc:1048] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:51.427006  1618 tablet_bootstrap.cc:492] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.426775  1685 consensus_queue.cc:1048] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:21:51.421468  1619 consensus_queue.cc:260] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.437449  1619 ts_tablet_manager.cc:1434] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:21:51.438683  1619 tablet_bootstrap.cc:492] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.438575  1677 raft_consensus.cc:493] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.439491  1677 raft_consensus.cc:515] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.444036  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:51.444801  1423 raft_consensus.cc:2468] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:51.445017  1677 leader_election.cc:290] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:51.446241  1463 leader_election.cc:304] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:51.447746  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.422073  1623 consensus_queue.cc:260] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.449118  1677 raft_consensus.cc:2804] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.449676  1677 raft_consensus.cc:493] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.450131  1677 raft_consensus.cc:3060] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.451294  1619 tablet_bootstrap.cc:654] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.448570  1573 raft_consensus.cc:2468] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:51.451406  1623 ts_tablet_manager.cc:1434] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:21:51.453930  1623 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.460453  1619 tablet_bootstrap.cc:492] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.461302  1677 raft_consensus.cc:515] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.462191  1619 ts_tablet_manager.cc:1403] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 12:21:51.464708  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:51.465400  1423 raft_consensus.cc:3060] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.465801  1619 raft_consensus.cc:359] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.466571  1619 raft_consensus.cc:385] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.466835  1619 raft_consensus.cc:740] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.469175  1622 tablet_bootstrap.cc:654] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.471956  1685 consensus_queue.cc:1048] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:51.475363  1618 tablet_bootstrap.cc:654] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.468080  1619 consensus_queue.cc:260] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.487429  1701 consensus_queue.cc:1048] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.491087  1619 ts_tablet_manager.cc:1434] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:21:51.492144  1619 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.497987  1619 tablet_bootstrap.cc:654] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.501935  1623 tablet_bootstrap.cc:654] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.503568  1622 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:51.504184  1622 ts_tablet_manager.cc:1403] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.080s	user 0.006s	sys 0.011s
I20260709 12:21:51.506637  1423 raft_consensus.cc:2468] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:51.509078  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:51.509052  1619 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.510908  1573 raft_consensus.cc:3060] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.509972  1677 leader_election.cc:290] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:51.512705  1619 ts_tablet_manager.cc:1403] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 12:21:51.513695  1618 tablet_bootstrap.cc:492] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.513943  1463 leader_election.cc:304] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:51.514259  1618 ts_tablet_manager.cc:1403] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.087s	user 0.010s	sys 0.006s
I20260709 12:21:51.515043  1677 raft_consensus.cc:2804] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.515471  1619 raft_consensus.cc:359] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.515807  1677 raft_consensus.cc:697] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:51.516216  1619 raft_consensus.cc:385] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.516592  1619 raft_consensus.cc:740] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.517277  1677 consensus_queue.cc:237] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.518754  1573 raft_consensus.cc:2468] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:51.519979  1619 consensus_queue.cc:260] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.520128  1622 raft_consensus.cc:359] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.520915  1622 raft_consensus.cc:385] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.521260  1622 raft_consensus.cc:740] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.521977  1619 ts_tablet_manager.cc:1434] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:51.522004  1622 consensus_queue.cc:260] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.523131  1619 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.523738  1618 raft_consensus.cc:359] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.524434  1618 raft_consensus.cc:385] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.524713  1618 raft_consensus.cc:740] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.525555  1618 consensus_queue.cc:260] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.526548  1622 ts_tablet_manager.cc:1434] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:21:51.528678  1622 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:51.529325  1618 ts_tablet_manager.cc:1434] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:21:51.530424  1618 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.533264  1623 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.533818  1623 ts_tablet_manager.cc:1403] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.004s
I20260709 12:21:51.536559  1623 raft_consensus.cc:359] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.537192  1623 raft_consensus.cc:385] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.537475  1623 raft_consensus.cc:740] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.538389  1618 tablet_bootstrap.cc:654] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.538275  1623 consensus_queue.cc:260] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.540803  1623 ts_tablet_manager.cc:1434] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:21:51.541950  1623 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.548166  1623 tablet_bootstrap.cc:654] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.548177  1244 catalog_manager.cc:5697] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.553359  1622 tablet_bootstrap.cc:654] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.564793  1685 raft_consensus.cc:493] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.565681  1619 tablet_bootstrap.cc:654] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.565495  1685 raft_consensus.cc:515] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.570295  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:51.571146  1498 raft_consensus.cc:2468] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 0.
I20260709 12:21:51.572481  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:51.572718  1312 leader_election.cc:304] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:51.573220  1423 raft_consensus.cc:2468] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 0.
I20260709 12:21:51.574012  1632 raft_consensus.cc:2804] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.567914  1685 leader_election.cc:290] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:51.574554  1632 raft_consensus.cc:493] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.575038  1632 raft_consensus.cc:3060] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.585389  1632 raft_consensus.cc:515] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.589262  1618 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.590108  1618 ts_tablet_manager.cc:1403] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.060s	user 0.020s	sys 0.009s
I20260709 12:21:51.590234  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:51.591217  1423 raft_consensus.cc:3060] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.600274  1618 raft_consensus.cc:359] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.601475  1618 raft_consensus.cc:385] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.601888  1618 raft_consensus.cc:740] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.603624  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:51.604378  1498 raft_consensus.cc:3060] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.605556  1622 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:51.606077  1622 ts_tablet_manager.cc:1403] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.008s
I20260709 12:21:51.605700  1618 consensus_queue.cc:260] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.608388  1618 ts_tablet_manager.cc:1434] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:21:51.608948  1622 raft_consensus.cc:359] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.609687  1622 raft_consensus.cc:385] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.609920  1618 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.612007  1623 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.612210  1498 raft_consensus.cc:2468] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:51.612551  1623 ts_tablet_manager.cc:1403] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.012s
I20260709 12:21:51.613633  1312 leader_election.cc:304] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:51.614537  1685 raft_consensus.cc:2804] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.614982  1685 raft_consensus.cc:697] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:21:51.615406  1623 raft_consensus.cc:359] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.616091  1623 raft_consensus.cc:385] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.615765  1685 consensus_queue.cc:237] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.616350  1623 raft_consensus.cc:740] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.619201  1623 consensus_queue.cc:260] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.617954  1618 tablet_bootstrap.cc:654] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.609970  1622 raft_consensus.cc:740] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.621094  1622 consensus_queue.cc:260] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.621785  1632 leader_election.cc:290] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:51.600841  1423 raft_consensus.cc:2468] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:51.624127  1677 raft_consensus.cc:493] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.624892  1677 raft_consensus.cc:515] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.627949  1622 ts_tablet_manager.cc:1434] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 12:21:51.628979  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.629429  1622 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:51.629899  1677 leader_election.cc:290] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
W20260709 12:21:51.630532  1461 leader_election.cc:343] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.630863  1619 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.631410  1619 ts_tablet_manager.cc:1403] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.108s	user 0.023s	sys 0.007s
I20260709 12:21:51.631454  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:51.632221  1423 raft_consensus.cc:2468] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:51.633566  1463 leader_election.cc:304] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:51.634347  1677 raft_consensus.cc:2804] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.634752  1677 raft_consensus.cc:493] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.635210  1677 raft_consensus.cc:3060] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.641160  1618 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.641853  1618 ts_tablet_manager.cc:1403] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.001s
I20260709 12:21:51.644976  1618 raft_consensus.cc:359] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.645656  1618 raft_consensus.cc:385] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.645917  1618 raft_consensus.cc:740] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.646701  1618 consensus_queue.cc:260] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.647467  1619 raft_consensus.cc:359] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.648212  1619 raft_consensus.cc:385] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.648502  1619 raft_consensus.cc:740] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.649102  1623 ts_tablet_manager.cc:1434] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:21:51.650261  1677 raft_consensus.cc:515] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.657109  1618 ts_tablet_manager.cc:1434] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:51.657651  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
W20260709 12:21:51.659194  1461 leader_election.cc:343] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.660305  1677 leader_election.cc:290] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:51.649293  1619 consensus_queue.cc:260] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.661399  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:51.662004  1423 raft_consensus.cc:3060] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.663262  1619 ts_tablet_manager.cc:1434] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 12:21:51.664593  1619 tablet_bootstrap.cc:492] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.666139  1677 raft_consensus.cc:493] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.666684  1670 raft_consensus.cc:493] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.668144  1623 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.671516  1670 raft_consensus.cc:515] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.673563  1423 raft_consensus.cc:2468] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:51.673839  1670 leader_election.cc:290] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.674866  1463 leader_election.cc:304] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:51.676069  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd76a0ea0cf343509a26d9f566ed01a1" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:51.676805  1498 raft_consensus.cc:2468] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:51.677174  1626 raft_consensus.cc:2804] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.677696  1626 raft_consensus.cc:697] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:51.677995  1623 tablet_bootstrap.cc:654] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.670264  1618 tablet_bootstrap.cc:492] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.678431  1537 leader_election.cc:304] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:51.679080  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd76a0ea0cf343509a26d9f566ed01a1" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.683295  1347 raft_consensus.cc:2468] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:51.678694  1626 consensus_queue.cc:237] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.684634  1670 raft_consensus.cc:2804] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.685050  1670 raft_consensus.cc:493] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.685413  1670 raft_consensus.cc:3060] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.689724  1618 tablet_bootstrap.cc:654] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.681262  1622 tablet_bootstrap.cc:654] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.690407  1685 raft_consensus.cc:493] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.691031  1685 raft_consensus.cc:515] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.679257  1677 raft_consensus.cc:515] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.693713  1677 leader_election.cc:290] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:51.678623  1244 catalog_manager.cc:5697] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 1 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.694070  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:51.694947  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
W20260709 12:21:51.711539  1461 leader_election.cc:343] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.695242  1685 leader_election.cc:290] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:51.696256  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:51.713021  1422 raft_consensus.cc:2468] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:51.696482  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.715639  1463 leader_election.cc:304] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:51.698668  1670 raft_consensus.cc:515] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.717697  1626 raft_consensus.cc:2804] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.718127  1626 raft_consensus.cc:493] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.718518  1626 raft_consensus.cc:3060] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.718881  1310 leader_election.cc:304] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:51.719080  1670 leader_election.cc:290] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.720263  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd76a0ea0cf343509a26d9f566ed01a1" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:51.720929  1347 raft_consensus.cc:3060] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.722097  1685 raft_consensus.cc:2749] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.709367  1619 tablet_bootstrap.cc:654] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.723466  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd76a0ea0cf343509a26d9f566ed01a1" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:51.724324  1498 raft_consensus.cc:3060] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.729101  1623 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.729606  1623 ts_tablet_manager.cc:1403] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.010s
I20260709 12:21:51.735616  1347 raft_consensus.cc:2468] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:51.738147  1623 raft_consensus.cc:359] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.737463  1243 catalog_manager.cc:5697] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.740685  1702 raft_consensus.cc:493] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.741248  1702 raft_consensus.cc:515] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.742511  1623 raft_consensus.cc:385] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.742828  1623 raft_consensus.cc:740] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.743443  1618 tablet_bootstrap.cc:492] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.743556  1702 leader_election.cc:290] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.744190  1618 ts_tablet_manager.cc:1403] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.009s
I20260709 12:21:51.745981  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6de610052e5443095393415714a09b3" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.746356  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6de610052e5443095393415714a09b3" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.747011  1573 raft_consensus.cc:2468] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
W20260709 12:21:51.747803  1386 leader_election.cc:343] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.740718  1622 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:51.748575  1386 leader_election.cc:304] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:51.749197  1622 ts_tablet_manager.cc:1403] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.015s
I20260709 12:21:51.750039  1619 tablet_bootstrap.cc:492] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.749897  1702 raft_consensus.cc:2804] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.750641  1702 raft_consensus.cc:493] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.743566  1623 consensus_queue.cc:260] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.751029  1702 raft_consensus.cc:3060] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.752022  1535 leader_election.cc:304] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:21:51.757684  1670 raft_consensus.cc:2804] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.758239  1619 ts_tablet_manager.cc:1403] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.087s	user 0.016s	sys 0.002s
I20260709 12:21:51.758937  1670 raft_consensus.cc:697] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:51.758829  1626 raft_consensus.cc:515] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.760004  1670 consensus_queue.cc:237] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.762652  1626 leader_election.cc:290] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:51.763305  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:51.749101  1618 raft_consensus.cc:359] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.762574  1623 ts_tablet_manager.cc:1434] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 12:21:51.767796  1618 raft_consensus.cc:385] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.763862  1702 raft_consensus.cc:515] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
W20260709 12:21:51.768303  1461 leader_election.cc:343] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.761840  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:51.769058  1422 raft_consensus.cc:3060] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.739068  1498 raft_consensus.cc:2468] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:51.770732  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6de610052e5443095393415714a09b3" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:51.771345  1573 raft_consensus.cc:3060] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.772173  1633 raft_consensus.cc:493] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.772634  1633 raft_consensus.cc:515] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.768190  1618 raft_consensus.cc:740] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.774708  1633 leader_election.cc:290] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.775405  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb6c9c94c1f54a0ab4c2e3266ce29f1c" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:51.775287  1618 consensus_queue.cc:260] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.776154  1498 raft_consensus.cc:2468] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:51.776311  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb6c9c94c1f54a0ab4c2e3266ce29f1c" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.777310  1537 leader_election.cc:304] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:51.777727  1618 ts_tablet_manager.cc:1434] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.002s
W20260709 12:21:51.777876  1535 leader_election.cc:343] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.778826  1618 tablet_bootstrap.cc:492] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.778936  1633 raft_consensus.cc:2804] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.779434  1633 raft_consensus.cc:493] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.779779  1633 raft_consensus.cc:3060] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.782779  1619 raft_consensus.cc:359] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.783573  1619 raft_consensus.cc:385] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.783885  1619 raft_consensus.cc:740] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.784593  1619 consensus_queue.cc:260] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.788355  1623 tablet_bootstrap.cc:492] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.789039  1422 raft_consensus.cc:2468] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:51.788007  1622 raft_consensus.cc:359] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.790172  1622 raft_consensus.cc:385] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.790612  1622 raft_consensus.cc:740] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.792878  1702 leader_election.cc:290] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.792781  1463 leader_election.cc:304] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:51.794281  1746 raft_consensus.cc:493] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.795696  1626 raft_consensus.cc:2804] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.796219  1626 raft_consensus.cc:697] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:51.796610  1619 ts_tablet_manager.cc:1434] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.038s	user 0.003s	sys 0.007s
I20260709 12:21:51.796465  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6de610052e5443095393415714a09b3" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:51.797063  1626 consensus_queue.cc:237] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:51.797953  1386 leader_election.cc:343] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.800314  1619 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.800256  1622 consensus_queue.cc:260] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.805094  1618 tablet_bootstrap.cc:654] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.799670  1746 raft_consensus.cc:515] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.793601  1633 raft_consensus.cc:515] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.812062  1573 raft_consensus.cc:2468] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:51.813110  1622 ts_tablet_manager.cc:1434] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.063s	user 0.002s	sys 0.005s
I20260709 12:21:51.819669  1622 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:51.819801  1746 leader_election.cc:290] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:51.820245  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c396f4bbb0fe496da0c70c8475f9a05e" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:51.820958  1497 raft_consensus.cc:2468] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:51.822439  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb6c9c94c1f54a0ab4c2e3266ce29f1c" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:51.824492  1388 leader_election.cc:304] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
W20260709 12:21:51.825901  1535 leader_election.cc:343] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.816524  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb6c9c94c1f54a0ab4c2e3266ce29f1c" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:51.817334  1633 leader_election.cc:290] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.826741  1498 raft_consensus.cc:3060] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.817660  1619 tablet_bootstrap.cc:654] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.830327  1386 leader_election.cc:304] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:51.832193  1622 tablet_bootstrap.cc:654] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.832306  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c396f4bbb0fe496da0c70c8475f9a05e" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.843813  1347 raft_consensus.cc:2468] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:51.833325  1240 catalog_manager.cc:5697] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.843951  1747 raft_consensus.cc:2804] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.846562  1747 raft_consensus.cc:493] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.846932  1747 raft_consensus.cc:3060] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.805713  1623 tablet_bootstrap.cc:654] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.857715  1619 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:51.858214  1619 ts_tablet_manager.cc:1403] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.013s
I20260709 12:21:51.858392  1747 raft_consensus.cc:515] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.862799  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c396f4bbb0fe496da0c70c8475f9a05e" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:51.863443  1497 raft_consensus.cc:3060] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.864498  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c396f4bbb0fe496da0c70c8475f9a05e" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:51.865216  1347 raft_consensus.cc:3060] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.868170  1702 raft_consensus.cc:2804] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.870599  1619 raft_consensus.cc:359] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.871320  1619 raft_consensus.cc:385] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.871598  1619 raft_consensus.cc:740] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.871629  1348 raft_consensus.cc:3060] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.872440  1619 consensus_queue.cc:260] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.883988  1702 raft_consensus.cc:697] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:51.885545  1702 consensus_queue.cc:237] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.882515  1746 raft_consensus.cc:493] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.899746  1746 raft_consensus.cc:515] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.907445  1623 tablet_bootstrap.cc:492] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:51.908030  1623 ts_tablet_manager.cc:1403] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.005s
I20260709 12:21:51.910473  1746 leader_election.cc:290] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.893456  1498 raft_consensus.cc:2468] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:51.912278  1537 leader_election.cc:304] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:51.913508  1633 raft_consensus.cc:2804] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.913601  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:51.913386  1243 catalog_manager.cc:5697] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.914084  1633 raft_consensus.cc:697] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
W20260709 12:21:51.904870  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:51.888017  1626 raft_consensus.cc:493] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.919896  1626 raft_consensus.cc:515] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.923496  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:51.925695  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "274048e07f194b339d084f93a0259790" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
W20260709 12:21:51.926136  1386 leader_election.cc:343] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.926424  1422 raft_consensus.cc:2468] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:51.887555  1747 leader_election.cc:290] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:51.927930  1633 consensus_queue.cc:237] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:51.928357  1463 leader_election.cc:304] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:51.929793  1625 raft_consensus.cc:2804] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.930190  1625 raft_consensus.cc:493] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.930615  1625 raft_consensus.cc:3060] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.933333  1622 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:51.934046  1622 ts_tablet_manager.cc:1403] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.115s	user 0.018s	sys 0.016s
I20260709 12:21:51.916052  1573 raft_consensus.cc:2468] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:51.937282  1622 raft_consensus.cc:359] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.937997  1622 raft_consensus.cc:385] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.918933  1347 raft_consensus.cc:2468] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:51.938264  1622 raft_consensus.cc:740] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.938287  1386 leader_election.cc:304] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:51.941879  1622 consensus_queue.cc:260] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.942159  1386 leader_election.cc:304] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:51.944434  1622 ts_tablet_manager.cc:1434] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:21:51.944500  1746 raft_consensus.cc:2804] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.944952  1746 raft_consensus.cc:493] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.945300  1746 raft_consensus.cc:3060] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.945725  1622 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:51.949761  1747 raft_consensus.cc:2804] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.929097  1497 raft_consensus.cc:2468] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:51.951882  1618 tablet_bootstrap.cc:492] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:51.952464  1618 ts_tablet_manager.cc:1403] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.174s	user 0.015s	sys 0.023s
I20260709 12:21:51.954591  1747 raft_consensus.cc:697] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:51.956568  1619 ts_tablet_manager.cc:1434] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.098s	user 0.022s	sys 0.016s
I20260709 12:21:51.956597  1747 consensus_queue.cc:237] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:51.958004  1619 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:51.962076  1746 raft_consensus.cc:515] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.966779  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:51.967366  1573 raft_consensus.cc:3060] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.967423  1623 raft_consensus.cc:359] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.968109  1623 raft_consensus.cc:385] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.968389  1623 raft_consensus.cc:740] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.976672  1619 tablet_bootstrap.cc:654] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.963120  1618 raft_consensus.cc:359] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.969767  1623 consensus_queue.cc:260] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.980486  1623 ts_tablet_manager.cc:1434] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.000s
I20260709 12:21:51.979617  1618 raft_consensus.cc:385] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.981365  1618 raft_consensus.cc:740] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.982321  1618 consensus_queue.cc:260] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.984658  1618 ts_tablet_manager.cc:1434] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:21:51.985189  1626 leader_election.cc:290] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:51.985962  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "274048e07f194b339d084f93a0259790" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
W20260709 12:21:51.988826  1461 leader_election.cc:343] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.990916  1708 raft_consensus.cc:493] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.991703  1746 leader_election.cc:290] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:51.991460  1708 raft_consensus.cc:515] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:51.990247  1623 tablet_bootstrap.cc:492] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:51.993638  1708 leader_election.cc:290] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:51.995482  1240 catalog_manager.cc:5697] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.999565  1622 tablet_bootstrap.cc:654] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.999750  1625 raft_consensus.cc:515] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:51.976430  1573 raft_consensus.cc:2468] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:52.004344  1618 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:51.999565  1623 tablet_bootstrap.cc:654] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.012024  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:52.012880  1572 raft_consensus.cc:2393] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 95782952bdee421094cada4ea955015d in this term.
I20260709 12:21:52.012969  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "274048e07f194b339d084f93a0259790" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:52.013700  1423 raft_consensus.cc:3060] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.010293  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:52.010771  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2efe5434a3fc4ba1a2552cc148e23a9a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:52.021080  1386 leader_election.cc:304] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:52.027963  1422 raft_consensus.cc:2393] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 95782952bdee421094cada4ea955015d in this term.
I20260709 12:21:52.029635  1746 raft_consensus.cc:2804] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.029856  1618 tablet_bootstrap.cc:654] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.022264  1625 leader_election.cc:290] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:52.030244  1746 raft_consensus.cc:697] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:52.032510  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "274048e07f194b339d084f93a0259790" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:52.037024  1312 leader_election.cc:304] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d
W20260709 12:21:52.037555  1461 leader_election.cc:343] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.024960  1348 raft_consensus.cc:3060] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.038753  1746 consensus_queue.cc:237] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.045524  1243 catalog_manager.cc:5697] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:52.047963  1625 consensus_queue.cc:1048] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.051699  1619 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:52.052269  1619 ts_tablet_manager.cc:1403] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.094s	user 0.007s	sys 0.010s
I20260709 12:21:52.056723  1348 raft_consensus.cc:2468] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:52.058106  1708 raft_consensus.cc:2749] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.070147  1618 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:52.072960  1618 ts_tablet_manager.cc:1403] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.004s
I20260709 12:21:52.074159  1619 raft_consensus.cc:359] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.074954  1619 raft_consensus.cc:385] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.075205  1619 raft_consensus.cc:740] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.080410  1702 raft_consensus.cc:493] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.081053  1702 raft_consensus.cc:515] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.081511  1423 raft_consensus.cc:2468] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:52.083027  1463 leader_election.cc:304] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:52.083392  1702 leader_election.cc:290] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:52.083174  1619 consensus_queue.cc:260] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.080405  1622 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:52.084738  1622 ts_tablet_manager.cc:1403] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.139s	user 0.014s	sys 0.004s
I20260709 12:21:52.085625  1625 raft_consensus.cc:2804] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.087000  1625 raft_consensus.cc:697] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:52.087805  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:52.087954  1625 consensus_queue.cc:237] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.089222  1670 raft_consensus.cc:493] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.088573  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:52.089790  1670 raft_consensus.cc:515] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.091006  1623 tablet_bootstrap.cc:492] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:52.091538  1623 ts_tablet_manager.cc:1403] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.024s
W20260709 12:21:52.091457  1386 leader_election.cc:343] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.092836  1618 raft_consensus.cc:359] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.093720  1670 leader_election.cc:290] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:52.094363  1623 raft_consensus.cc:359] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.095726  1618 raft_consensus.cc:385] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.096352  1623 raft_consensus.cc:385] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.096599  1618 raft_consensus.cc:740] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.096833  1623 raft_consensus.cc:740] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.097806  1618 consensus_queue.cc:260] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.097844  1623 consensus_queue.cc:260] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.100749  1623 ts_tablet_manager.cc:1434] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
W20260709 12:21:52.096112  1388 leader_election.cc:343] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.103402  1623 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:52.105687  1619 ts_tablet_manager.cc:1434] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.005s
I20260709 12:21:52.107206  1619 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:52.109577  1618 ts_tablet_manager.cc:1434] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
I20260709 12:21:52.096544  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:52.111492  1618 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:52.125094  1623 tablet_bootstrap.cc:654] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.124683  1677 consensus_queue.cc:1048] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.124766  1743 consensus_queue.cc:1048] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.129788  1388 leader_election.cc:304] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:52.130848  1702 raft_consensus.cc:2749] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.115164  1497 raft_consensus.cc:2468] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:52.115584  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:52.135943  1535 leader_election.cc:343] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.135972  1622 raft_consensus.cc:359] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.138038  1622 raft_consensus.cc:385] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.138584  1622 raft_consensus.cc:740] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.157392  1622 consensus_queue.cc:260] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.163062  1618 tablet_bootstrap.cc:654] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.170799  1623 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:52.171406  1623 ts_tablet_manager.cc:1403] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.007s
I20260709 12:21:52.173755  1622 ts_tablet_manager.cc:1434] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.000s
I20260709 12:21:52.175168  1622 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:52.156620  1537 leader_election.cc:304] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:52.176614  1632 consensus_queue.cc:1048] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:52.180799  1619 tablet_bootstrap.cc:654] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.171888  1240 catalog_manager.cc:5697] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:52.182660  1670 raft_consensus.cc:2804] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.183064  1670 raft_consensus.cc:493] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.183400  1670 raft_consensus.cc:3060] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:52.176863  1461 consensus_peers.cc:597] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.185454  1243 catalog_manager.cc:5697] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:52.203455  1670 raft_consensus.cc:515] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.204718  1622 tablet_bootstrap.cc:654] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.207293  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:52.207758  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:52.208415  1497 raft_consensus.cc:3060] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:52.208935  1535 leader_election.cc:343] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.209110  1670 leader_election.cc:290] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:52.208854  1618 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:52.195703  1632 raft_consensus.cc:493] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.210022  1618 ts_tablet_manager.cc:1403] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.099s	user 0.022s	sys 0.011s
I20260709 12:21:52.209940  1632 raft_consensus.cc:515] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.212226  1632 leader_election.cc:290] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:52.213146  1618 raft_consensus.cc:359] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.213842  1618 raft_consensus.cc:385] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.214103  1618 raft_consensus.cc:740] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.214830  1618 consensus_queue.cc:260] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.216113  1497 raft_consensus.cc:2468] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:52.216770  1618 ts_tablet_manager.cc:1434] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:52.217376  1537 leader_election.cc:304] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:52.217859  1618 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:52.218379  1670 raft_consensus.cc:2804] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.218932  1670 raft_consensus.cc:697] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:52.219731  1670 consensus_queue.cc:237] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.226054  1244 catalog_manager.cc:5697] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:52.240415  1618 tablet_bootstrap.cc:654] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.240198  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb6c9c94c1f54a0ab4c2e3266ce29f1c" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:52.249614  1622 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:52.250173  1622 ts_tablet_manager.cc:1403] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.075s	user 0.008s	sys 0.004s
I20260709 12:21:52.252439  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb6c9c94c1f54a0ab4c2e3266ce29f1c" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:52.253244  1498 raft_consensus.cc:2393] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
I20260709 12:21:52.253134  1622 raft_consensus.cc:359] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.253978  1622 raft_consensus.cc:385] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.254302  1622 raft_consensus.cc:740] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.258111  1774 consensus_queue.cc:1048] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.259044  1633 consensus_queue.cc:1048] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.258296  1312 leader_election.cc:304] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:52.255136  1622 consensus_queue.cc:260] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.270962  1632 raft_consensus.cc:3060] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.299649  1240 catalog_manager.cc:5697] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:52.281481  1619 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:52.284191  1633 raft_consensus.cc:493] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.291816  1702 raft_consensus.cc:493] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.304116  1622 ts_tablet_manager.cc:1434] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 12:21:52.304330  1619 ts_tablet_manager.cc:1403] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.197s	user 0.016s	sys 0.003s
I20260709 12:21:52.278313  1623 raft_consensus.cc:359] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.303761  1633 raft_consensus.cc:515] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.308151  1622 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:52.308220  1623 raft_consensus.cc:385] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.312608  1623 raft_consensus.cc:740] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.313567  1623 consensus_queue.cc:260] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.317425  1633 leader_election.cc:290] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:52.317646  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:52.307407  1702 raft_consensus.cc:515] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.324689  1623 ts_tablet_manager.cc:1434] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.153s	user 0.007s	sys 0.003s
I20260709 12:21:52.323024  1702 leader_election.cc:290] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:52.325752  1623 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:52.327144  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2db5d455eb64aad9a2ab7d8fbf9eeef" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:52.327564  1618 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:52.328125  1618 ts_tablet_manager.cc:1403] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.110s	user 0.017s	sys 0.012s
W20260709 12:21:52.328823  1386 leader_election.cc:343] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.322464  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:52.323382  1424 raft_consensus.cc:2393] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76217635dae3410b96baf380d386cbee in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:52.335879  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2db5d455eb64aad9a2ab7d8fbf9eeef" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:52.336239  1537 leader_election.cc:304] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:52.336658  1572 raft_consensus.cc:2468] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:52.337795  1386 leader_election.cc:304] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:52.331166  1618 raft_consensus.cc:359] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.338023  1625 raft_consensus.cc:493] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.338573  1618 raft_consensus.cc:385] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.339017  1618 raft_consensus.cc:740] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.339325  1619 raft_consensus.cc:359] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.340078  1619 raft_consensus.cc:385] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.339774  1618 consensus_queue.cc:260] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.340395  1619 raft_consensus.cc:740] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.341192  1619 consensus_queue.cc:260] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.344449  1780 raft_consensus.cc:3060] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.344502  1626 consensus_queue.cc:1048] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.355808  1632 raft_consensus.cc:2749] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.338678  1625 raft_consensus.cc:515] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.361661  1625 leader_election.cc:290] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:52.361904  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6baed42964ae8a9fbab7c6e8601c0" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
W20260709 12:21:52.363353  1463 leader_election.cc:343] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.347851  1622 tablet_bootstrap.cc:654] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.348260  1623 tablet_bootstrap.cc:654] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.352042  1618 ts_tablet_manager.cc:1434] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 12:21:52.371665  1618 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:52.352138  1747 raft_consensus.cc:2804] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.374687  1747 raft_consensus.cc:493] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.375110  1747 raft_consensus.cc:3060] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.353855  1619 ts_tablet_manager.cc:1434] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:21:52.379211  1619 tablet_bootstrap.cc:492] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:52.385082  1783 consensus_queue.cc:1048] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:21:52.408484  1618 tablet_bootstrap.cc:654] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.416937  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6baed42964ae8a9fbab7c6e8601c0" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:52.417696  1573 raft_consensus.cc:2468] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:52.417762  1780 raft_consensus.cc:2749] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.420521  1702 consensus_queue.cc:1048] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.421895  1702 consensus_queue.cc:1048] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.423466  1461 leader_election.cc:304] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 95782952bdee421094cada4ea955015d
I20260709 12:21:52.425930  1743 raft_consensus.cc:2804] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.426458  1743 raft_consensus.cc:493] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.428372  1619 tablet_bootstrap.cc:654] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.427627  1743 raft_consensus.cc:3060] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.435767  1622 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:52.436475  1622 ts_tablet_manager.cc:1403] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.011s
I20260709 12:21:52.450389  1747 raft_consensus.cc:515] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.452811  1747 leader_election.cc:290] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:52.456949  1625 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.457731  1625 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.465641  1625 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:52.465662  1622 raft_consensus.cc:359] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.467155  1622 raft_consensus.cc:385] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.467613  1622 raft_consensus.cc:740] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.469247  1622 consensus_queue.cc:260] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.454957  1632 raft_consensus.cc:493] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.475703  1625 raft_consensus.cc:493] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.479229  1625 raft_consensus.cc:515] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.479528  1622 ts_tablet_manager.cc:1434] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 12:21:52.481319  1622 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:52.482779  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a329febbdb4a35accb4f17719726dc" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:52.483469  1424 raft_consensus.cc:2468] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:52.484899  1463 leader_election.cc:304] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:52.486114  1626 raft_consensus.cc:2804] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.486476  1626 raft_consensus.cc:493] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.486811  1626 raft_consensus.cc:3060] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.488598  1622 tablet_bootstrap.cc:654] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.492599  1632 raft_consensus.cc:515] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.497568  1632 leader_election.cc:290] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
W20260709 12:21:52.500139  1386 consensus_peers.cc:597] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.500231  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2db5d455eb64aad9a2ab7d8fbf9eeef" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:52.503177  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:52.504765  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:52.505198  1702 consensus_queue.cc:1048] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.505517  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:52.506734  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2db5d455eb64aad9a2ab7d8fbf9eeef" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:52.507355  1572 raft_consensus.cc:3060] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.508018  1783 consensus_queue.cc:1048] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:52.511480  1461 leader_election.cc:343] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.481359  1625 leader_election.cc:290] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:52.513019  1625 consensus_queue.cc:1048] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.514544  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a329febbdb4a35accb4f17719726dc" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:52.516254  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:52.517002  1461 leader_election.cc:343] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:52.518036  1461 leader_election.cc:343] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.522410  1312 leader_election.cc:304] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:52.546376  1632 raft_consensus.cc:2749] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.552500  1623 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:52.553234  1623 ts_tablet_manager.cc:1403] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.228s	user 0.025s	sys 0.014s
W20260709 12:21:52.521731  1386 leader_election.cc:343] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.519815  1626 raft_consensus.cc:515] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.518464  1461 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:52.564800  1747 consensus_queue.cc:1048] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.520388  1743 raft_consensus.cc:515] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:52.577812  1461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 59789.8us
I20260709 12:21:52.578960  1677 raft_consensus.cc:2749] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.568878  1746 consensus_queue.cc:1048] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:21:52.580847  1625 consensus_queue.cc:1048] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.583499  1743 leader_election.cc:290] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:52.591377  1626 leader_election.cc:290] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:52.567955  1632 raft_consensus.cc:493] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.592756  1632 raft_consensus.cc:515] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.594789  1632 leader_election.cc:290] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:52.598148  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:52.606763  1626 raft_consensus.cc:493] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.610391  1572 raft_consensus.cc:2468] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:52.619097  1496 raft_consensus.cc:2393] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
I20260709 12:21:52.619941  1626 raft_consensus.cc:515] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.629259  1618 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:52.630808  1618 ts_tablet_manager.cc:1403] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.259s	user 0.010s	sys 0.019s
I20260709 12:21:52.624120  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61b61c6a29814aad84e50fcb39c26e56" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
W20260709 12:21:52.623054  1463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 43995us
I20260709 12:21:52.634058  1619 tablet_bootstrap.cc:492] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:52.634662  1622 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:52.640612  1622 ts_tablet_manager.cc:1403] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.018s
I20260709 12:21:52.636864  1386 leader_election.cc:304] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:52.644672  1747 raft_consensus.cc:2804] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.645174  1747 raft_consensus.cc:697] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:52.645920  1747 consensus_queue.cc:237] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.646081  1618 raft_consensus.cc:359] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.639046  1310 leader_election.cc:304] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:52.646998  1618 raft_consensus.cc:385] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.648597  1774 raft_consensus.cc:3060] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.648995  1618 raft_consensus.cc:740] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.650116  1618 consensus_queue.cc:260] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.654191  1618 ts_tablet_manager.cc:1434] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:21:52.654970  1623 raft_consensus.cc:359] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.655721  1623 raft_consensus.cc:385] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.655988  1623 raft_consensus.cc:740] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.657428  1619 ts_tablet_manager.cc:1403] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.279s	user 0.025s	sys 0.018s
I20260709 12:21:52.659315  1618 tablet_bootstrap.cc:492] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:52.660686  1623 consensus_queue.cc:260] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.671541  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6baed42964ae8a9fbab7c6e8601c0" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:52.672583  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a329febbdb4a35accb4f17719726dc" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:52.673293  1422 raft_consensus.cc:3060] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.674228  1626 leader_election.cc:290] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:52.678931  1623 ts_tablet_manager.cc:1434] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.125s	user 0.008s	sys 0.001s
I20260709 12:21:52.679162  1622 raft_consensus.cc:359] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:52.680444  1622 raft_consensus.cc:385] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.680958  1622 raft_consensus.cc:740] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.681830  1622 consensus_queue.cc:260] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
W20260709 12:21:52.683936  1463 leader_election.cc:343] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:52.632159  1461 consensus_peers.cc:597] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 -> Peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Couldn't send request to peer 76217635dae3410b96baf380d386cbee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:52.686183  1461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 58233.4us
I20260709 12:21:52.687866  1622 ts_tablet_manager.cc:1434] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.000s
I20260709 12:21:52.688758  1623 tablet_bootstrap.cc:492] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:52.692778  1622 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:52.689461  1619 raft_consensus.cc:359] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.693434  1619 raft_consensus.cc:385] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.693897  1619 raft_consensus.cc:740] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.691778  1422 raft_consensus.cc:2468] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:52.691845  1774 raft_consensus.cc:2749] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.701932  1619 consensus_queue.cc:260] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.703809  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:52.702462  1626 raft_consensus.cc:493] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.704762  1423 raft_consensus.cc:2468] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:52.706260  1626 raft_consensus.cc:515] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:52.705724  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6baed42964ae8a9fbab7c6e8601c0" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:52.712414  1626 leader_election.cc:290] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:52.716017  1348 raft_consensus.cc:3060] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.716233  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a329febbdb4a35accb4f17719726dc" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:52.720009  1780 consensus_queue.cc:1048] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:52.728538  1461 leader_election.cc:343] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.729945  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52095deeb4984630a0d2b2ab844fef7d" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:52.730091  1463 leader_election.cc:304] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:52.732795  1797 raft_consensus.cc:2804] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.733217  1797 raft_consensus.cc:493] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.733610  1797 raft_consensus.cc:3060] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.734680  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52095deeb4984630a0d2b2ab844fef7d" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:52.737357  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:52.737382  1463 leader_election.cc:304] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:52.738806  1625 raft_consensus.cc:2804] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.707561  1570 raft_consensus.cc:3060] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.739475  1625 raft_consensus.cc:697] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:52.751214  1625 consensus_queue.cc:237] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.753909  1780 consensus_queue.cc:1048] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.754313  1243 catalog_manager.cc:5697] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:52.755921  1623 tablet_bootstrap.cc:654] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.759244  1622 tablet_bootstrap.cc:654] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:52.751649  1461 leader_election.cc:343] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:52.765494  1461 leader_election.cc:343] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:52.766765  1461 leader_election.cc:343] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.769938  1619 ts_tablet_manager.cc:1434] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.001s
I20260709 12:21:52.772178  1619 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:52.782966  1797 raft_consensus.cc:515] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.787578  1797 leader_election.cc:290] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:52.788722  1618 tablet_bootstrap.cc:654] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.789731  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:52.795292  1424 raft_consensus.cc:3060] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.812086  1618 tablet_bootstrap.cc:492] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:52.812983  1618 ts_tablet_manager.cc:1403] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.158s	user 0.015s	sys 0.000s
I20260709 12:21:52.815111  1424 raft_consensus.cc:2468] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:52.816802  1463 leader_election.cc:304] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:52.818153  1797 raft_consensus.cc:2804] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.818924  1797 raft_consensus.cc:697] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:52.820348  1797 consensus_queue.cc:237] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:52.775262  1461 leader_election.cc:304] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:52.826758  1626 raft_consensus.cc:2749] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.829295  1618 raft_consensus.cc:359] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.831401  1618 raft_consensus.cc:385] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.831708  1618 raft_consensus.cc:740] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.802130  1570 raft_consensus.cc:2468] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:52.832459  1618 consensus_queue.cc:260] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.835835  1618 ts_tablet_manager.cc:1434] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.002s
W20260709 12:21:52.827260  1461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 60237.5us
I20260709 12:21:52.841738  1348 raft_consensus.cc:3060] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.843320  1461 leader_election.cc:304] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 95782952bdee421094cada4ea955015d
I20260709 12:21:52.843956  1618 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:52.844120  1626 raft_consensus.cc:2804] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.844749  1626 raft_consensus.cc:697] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:52.845892  1626 consensus_queue.cc:237] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.850507  1618 tablet_bootstrap.cc:654] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.854986  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:52.857872  1619 tablet_bootstrap.cc:654] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:52.861624  1461 leader_election.cc:343] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.864336  1623 tablet_bootstrap.cc:492] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:52.864914  1623 ts_tablet_manager.cc:1403] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.178s	user 0.011s	sys 0.004s
I20260709 12:21:52.864156  1245 catalog_manager.cc:5697] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:52.878010  1670 consensus_queue.cc:1048] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.896150  1747 raft_consensus.cc:493] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.896713  1747 raft_consensus.cc:515] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.897509  1623 raft_consensus.cc:359] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.898222  1623 raft_consensus.cc:385] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.899796  1623 raft_consensus.cc:740] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.901216  1623 consensus_queue.cc:260] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:52.901600  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6baed42964ae8a9fbab7c6e8601c0" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:52.903453  1623 ts_tablet_manager.cc:1434] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:21:52.904464  1623 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:52.907492  1747 leader_election.cc:290] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:52.912331  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6baed42964ae8a9fbab7c6e8601c0" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:52.914737  1570 raft_consensus.cc:2393] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95782952bdee421094cada4ea955015d in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:52.931454  1797 raft_consensus.cc:493] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.932412  1797 raft_consensus.cc:515] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.022583  1386 leader_election.cc:304] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:53.023571  1747 raft_consensus.cc:3060] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:53.023715  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 107790us
I20260709 12:21:53.038286  1622 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:53.038861  1622 ts_tablet_manager.cc:1403] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.346s	user 0.016s	sys 0.014s
I20260709 12:21:53.047442  1623 tablet_bootstrap.cc:654] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.050068  1619 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:53.050734  1619 ts_tablet_manager.cc:1403] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.279s	user 0.010s	sys 0.015s
I20260709 12:21:53.052250  1747 raft_consensus.cc:2749] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:53.055390  1618 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:53.055629  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85193c93a364d41b3124bde25907dee" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:53.055951  1618 ts_tablet_manager.cc:1403] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.216s	user 0.023s	sys 0.028s
W20260709 12:21:53.057145  1463 leader_election.cc:343] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.057965  1797 leader_election.cc:290] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:53.059937  1780 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.060477  1780 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.062772  1780 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:53.065912  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.066288  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85193c93a364d41b3124bde25907dee" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:53.067819  1498 raft_consensus.cc:2468] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
W20260709 12:21:53.068404  1461 leader_election.cc:343] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.069000  1461 leader_election.cc:304] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d
I20260709 12:21:53.072299  1244 catalog_manager.cc:5697] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:53.080235  1244 catalog_manager.cc:5697] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:53.087944  1619 raft_consensus.cc:359] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.088650  1623 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:53.089182  1623 ts_tablet_manager.cc:1403] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.185s	user 0.012s	sys 0.000s
I20260709 12:21:53.073156  1797 raft_consensus.cc:2749] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:53.091221  1622 raft_consensus.cc:359] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.091935  1622 raft_consensus.cc:385] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.092191  1622 raft_consensus.cc:740] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.092872  1622 consensus_queue.cc:260] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.074210  1537 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:53.095652  1670 raft_consensus.cc:2804] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.096122  1670 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.096508  1670 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.073173  1618 raft_consensus.cc:359] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.099071  1618 raft_consensus.cc:385] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.099385  1618 raft_consensus.cc:740] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.075829  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:53.101291  1535 leader_election.cc:343] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.102761  1618 consensus_queue.cc:260] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.104573  1571 raft_consensus.cc:3060] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.107204  1618 ts_tablet_manager.cc:1434] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.001s
I20260709 12:21:53.108134  1618 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:53.110975  1670 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.088661  1619 raft_consensus.cc:385] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.114444  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:53.115078  1619 raft_consensus.cc:740] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.115769  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
W20260709 12:21:53.116330  1535 leader_election.cc:343] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.116387  1497 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.117527  1619 consensus_queue.cc:260] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.123663  1497 raft_consensus.cc:2468] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:53.117025  1670 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:53.127662  1537 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:53.129323  1619 ts_tablet_manager.cc:1434] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.001s
I20260709 12:21:53.135936  1619 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:53.130667  1622 ts_tablet_manager.cc:1434] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.091s	user 0.006s	sys 0.001s
I20260709 12:21:53.131994  1623 raft_consensus.cc:359] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.139818  1623 raft_consensus.cc:385] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.140123  1623 raft_consensus.cc:740] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.141167  1618 tablet_bootstrap.cc:654] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.141291  1780 raft_consensus.cc:2804] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.141825  1780 raft_consensus.cc:697] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:53.142747  1780 consensus_queue.cc:237] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.138833  1622 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:53.144696  1623 consensus_queue.cc:260] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.152736  1623 ts_tablet_manager.cc:1434] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.001s
I20260709 12:21:53.154023  1623 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
W20260709 12:21:53.160921  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:53.165355  1747 consensus_queue.cc:1048] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.173801  1623 tablet_bootstrap.cc:654] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.175438  1622 tablet_bootstrap.cc:654] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.176676  1618 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:53.175644  1619 tablet_bootstrap.cc:654] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.179927  1245 catalog_manager.cc:5697] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:53.180969  1618 ts_tablet_manager.cc:1403] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.073s	user 0.008s	sys 0.004s
I20260709 12:21:53.181934  1348 raft_consensus.cc:3060] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.184993  1618 raft_consensus.cc:359] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.191166  1618 raft_consensus.cc:385] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.191555  1618 raft_consensus.cc:740] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.192386  1618 consensus_queue.cc:260] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.195314  1746 consensus_queue.cc:1048] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.204062  1619 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:53.204596  1619 ts_tablet_manager.cc:1403] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.007s
I20260709 12:21:53.207270  1619 raft_consensus.cc:359] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.207916  1619 raft_consensus.cc:385] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.208185  1619 raft_consensus.cc:740] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.209031  1619 consensus_queue.cc:260] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.220018  1619 ts_tablet_manager.cc:1434] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:21:53.224066  1619 tablet_bootstrap.cc:492] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:53.229971  1619 tablet_bootstrap.cc:654] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.237805  1618 ts_tablet_manager.cc:1434] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.001s
I20260709 12:21:53.241255  1618 tablet_bootstrap.cc:492] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:53.245048  1622 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:53.245563  1622 ts_tablet_manager.cc:1403] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.107s	user 0.009s	sys 0.030s
I20260709 12:21:53.245900  1619 tablet_bootstrap.cc:492] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:53.246469  1619 ts_tablet_manager.cc:1403] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.010s
I20260709 12:21:53.248100  1622 raft_consensus.cc:359] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.248512  1623 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:53.248911  1622 raft_consensus.cc:385] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.249218  1622 raft_consensus.cc:740] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.249351  1623 ts_tablet_manager.cc:1403] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.096s	user 0.029s	sys 0.009s
I20260709 12:21:53.249434  1619 raft_consensus.cc:359] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.250304  1619 raft_consensus.cc:385] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.250000  1622 consensus_queue.cc:260] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.250636  1619 raft_consensus.cc:740] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.251499  1619 consensus_queue.cc:260] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.252518  1623 raft_consensus.cc:359] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.253265  1623 raft_consensus.cc:385] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.253957  1623 raft_consensus.cc:740] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.254721  1623 consensus_queue.cc:260] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.260493  1623 ts_tablet_manager.cc:1434] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 12:21:53.248498  1618 tablet_bootstrap.cc:654] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.261580  1623 tablet_bootstrap.cc:492] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:53.264233  1622 ts_tablet_manager.cc:1434] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 12:21:53.265184  1622 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:53.267041  1796 consensus_queue.cc:1048] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:53.272112  1747 raft_consensus.cc:493] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.273011  1619 ts_tablet_manager.cc:1434] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.003s
I20260709 12:21:53.274168  1619 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:53.274901  1622 tablet_bootstrap.cc:654] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:53.276556  1461 consensus_peers.cc:597] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:53.272655  1747 raft_consensus.cc:515] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.279671  1623 tablet_bootstrap.cc:654] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.279798  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0e5cd0b264bb18307cc2498883adf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:53.280797  1570 raft_consensus.cc:2468] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:53.282123  1386 leader_election.cc:304] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:53.283095  1746 raft_consensus.cc:2804] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.283502  1746 raft_consensus.cc:493] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.283847  1746 raft_consensus.cc:3060] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.295861  1619 tablet_bootstrap.cc:654] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.297080  1623 tablet_bootstrap.cc:492] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:53.297850  1623 ts_tablet_manager.cc:1403] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.010s
I20260709 12:21:53.300952  1622 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:53.301534  1622 ts_tablet_manager.cc:1403] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.009s
I20260709 12:21:53.301003  1623 raft_consensus.cc:359] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.302021  1623 raft_consensus.cc:385] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.302368  1623 raft_consensus.cc:740] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.303723  1623 consensus_queue.cc:260] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.304823  1622 raft_consensus.cc:359] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.305585  1622 raft_consensus.cc:385] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.305862  1622 raft_consensus.cc:740] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.307719  1345 raft_consensus.cc:3060] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.308632  1619 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:53.309231  1619 ts_tablet_manager.cc:1403] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.000s
I20260709 12:21:53.311754  1622 consensus_queue.cc:260] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.314173  1623 ts_tablet_manager.cc:1434] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.004s
I20260709 12:21:53.315311  1623 tablet_bootstrap.cc:492] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:53.319447  1747 leader_election.cc:290] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:53.323024  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0e5cd0b264bb18307cc2498883adf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:53.324366  1796 consensus_queue.cc:1048] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:53.327659  1386 leader_election.cc:343] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.329686  1622 ts_tablet_manager.cc:1434] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 12:21:53.336196  1622 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:53.336557  1623 tablet_bootstrap.cc:654] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.349114  1623 tablet_bootstrap.cc:492] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:53.349593  1796 raft_consensus.cc:493] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.349596  1622 tablet_bootstrap.cc:654] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.349918  1623 ts_tablet_manager.cc:1403] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.012s
I20260709 12:21:53.352144  1619 raft_consensus.cc:359] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.353222  1619 raft_consensus.cc:385] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.353535  1619 raft_consensus.cc:740] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.354579  1619 consensus_queue.cc:260] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.357048  1619 ts_tablet_manager.cc:1434] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.003s
I20260709 12:21:53.358315  1619 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:53.356307  1623 raft_consensus.cc:359] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.351488  1801 raft_consensus.cc:493] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.350811  1796 raft_consensus.cc:515] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.361501  1623 raft_consensus.cc:385] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.361907  1623 raft_consensus.cc:740] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.362983  1801 raft_consensus.cc:515] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.366118  1623 consensus_queue.cc:260] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.379766  1623 ts_tablet_manager.cc:1434] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:21:53.380949  1796 leader_election.cc:290] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:53.386353  1780 raft_consensus.cc:493] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.387153  1780 raft_consensus.cc:515] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.389448  1774 raft_consensus.cc:493] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.389736  1780 leader_election.cc:290] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:53.390245  1774 raft_consensus.cc:515] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.396378  1623 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:53.405886  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:53.406440  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:53.407874  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:53.408520  1423 raft_consensus.cc:2468] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:53.410698  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52095deeb4984630a0d2b2ab844fef7d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:21:53.411093  1461 leader_election.cc:343] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.410967  1796 consensus_queue.cc:1048] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:53.411303  1497 raft_consensus.cc:2468] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:53.412986  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52095deeb4984630a0d2b2ab844fef7d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:53.414170  1386 leader_election.cc:343] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.415077  1747 raft_consensus.cc:493] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:53.415644  1535 leader_election.cc:343] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.415607  1747 raft_consensus.cc:515] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.418546  1537 leader_election.cc:304] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:53.419337  1463 leader_election.cc:304] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:53.420817  1670 raft_consensus.cc:2804] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.421396  1670 raft_consensus.cc:493] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.393637  1801 leader_election.cc:290] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:53.421530  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85193c93a364d41b3124bde25907dee" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.422391  1498 raft_consensus.cc:2468] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:53.421860  1670 raft_consensus.cc:3060] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.424207  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.418872  1747 leader_election.cc:290] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:53.420904  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85193c93a364d41b3124bde25907dee" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:53.430027  1570 raft_consensus.cc:2468] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:53.421514  1797 raft_consensus.cc:2804] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.430821  1797 raft_consensus.cc:493] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.431233  1797 raft_consensus.cc:3060] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.436343  1386 leader_election.cc:304] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:53.437597  1747 raft_consensus.cc:2804] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.438063  1747 raft_consensus.cc:493] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.438444  1747 raft_consensus.cc:3060] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.439972  1623 tablet_bootstrap.cc:654] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.444883  1619 tablet_bootstrap.cc:654] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.461735  1625 consensus_queue.cc:1048] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:53.467253  1626 consensus_queue.cc:1048] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.451408  1746 raft_consensus.cc:515] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.471611  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0e5cd0b264bb18307cc2498883adf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:53.472496  1570 raft_consensus.cc:3060] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.476022  1622 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:53.476562  1622 ts_tablet_manager.cc:1403] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.141s	user 0.027s	sys 0.018s
I20260709 12:21:53.480036  1747 raft_consensus.cc:515] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.484433  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:53.499874  1622 raft_consensus.cc:359] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.501184  1622 raft_consensus.cc:385] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.501487  1622 raft_consensus.cc:740] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.502950  1622 consensus_queue.cc:260] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.513417  1774 leader_election.cc:290] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:53.516755  1618 tablet_bootstrap.cc:492] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:53.517506  1618 ts_tablet_manager.cc:1403] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.277s	user 0.022s	sys 0.017s
I20260709 12:21:53.480717  1670 raft_consensus.cc:515] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.531821  1622 ts_tablet_manager.cc:1434] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.055s	user 0.012s	sys 0.003s
I20260709 12:21:53.535182  1746 leader_election.cc:290] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
W20260709 12:21:53.521293  1463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 54263us
I20260709 12:21:53.536365  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0e5cd0b264bb18307cc2498883adf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:53.546252  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.547171  1623 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
W20260709 12:21:53.548398  1386 leader_election.cc:343] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.548379  1623 ts_tablet_manager.cc:1403] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.154s	user 0.019s	sys 0.007s
I20260709 12:21:53.553013  1747 leader_election.cc:290] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:53.553246  1312 leader_election.cc:304] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:53.516320  1797 raft_consensus.cc:515] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.558122  1622 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:53.564987  1497 raft_consensus.cc:2393] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95782952bdee421094cada4ea955015d in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:53.565596  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85193c93a364d41b3124bde25907dee" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:53.566255  1569 raft_consensus.cc:3060] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.573396  1570 raft_consensus.cc:2468] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:53.573766  1569 raft_consensus.cc:2468] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:53.575614  1618 raft_consensus.cc:359] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.576167  1618 raft_consensus.cc:385] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.576404  1618 raft_consensus.cc:740] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.576951  1618 consensus_queue.cc:260] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:53.634421  1312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 81035.5us
I20260709 12:21:53.640657  1386 leader_election.cc:304] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
W20260709 12:21:53.647161  1461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 74989.2us
I20260709 12:21:53.654632  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:53.655944  1850 raft_consensus.cc:2749] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:53.657615  1461 leader_election.cc:343] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.665151  1619 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:53.666150  1618 ts_tablet_manager.cc:1434] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.148s	user 0.011s	sys 0.001s
I20260709 12:21:53.666596  1619 ts_tablet_manager.cc:1403] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.309s	user 0.024s	sys 0.012s
I20260709 12:21:53.593427  1797 leader_election.cc:290] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:53.674643  1670 leader_election.cc:290] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:53.708200  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52095deeb4984630a0d2b2ab844fef7d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
W20260709 12:21:53.708436  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 68416.1us
W20260709 12:21:53.711134  1535 leader_election.cc:343] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.712828  1746 raft_consensus.cc:2804] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.713583  1746 raft_consensus.cc:697] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:53.715684  1746 consensus_queue.cc:237] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.717300  1841 raft_consensus.cc:493] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.718354  1841 raft_consensus.cc:515] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.723229  1348 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.723790  1841 leader_election.cc:290] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:53.737526  1622 tablet_bootstrap.cc:654] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.740875  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a329febbdb4a35accb4f17719726dc" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.731122  1801 raft_consensus.cc:493] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.744068  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52095deeb4984630a0d2b2ab844fef7d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:53.771474  1670 raft_consensus.cc:493] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.772368  1670 raft_consensus.cc:515] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.711534  1386 leader_election.cc:304] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:53.794945  1801 raft_consensus.cc:515] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:53.798130  1801 leader_election.cc:290] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:53.800154  1618 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:53.801512  1497 raft_consensus.cc:3060] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.816236  1622 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:53.817399  1622 ts_tablet_manager.cc:1403] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.260s	user 0.015s	sys 0.012s
I20260709 12:21:53.816996  1619 raft_consensus.cc:359] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.814074  1670 leader_election.cc:290] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:53.784160  1623 raft_consensus.cc:359] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.818341  1619 raft_consensus.cc:385] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.827116  1623 raft_consensus.cc:385] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.827276  1619 raft_consensus.cc:740] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
W20260709 12:21:53.828295  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 117498us
I20260709 12:21:53.828822  1619 consensus_queue.cc:260] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.831180  1747 raft_consensus.cc:2804] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.829902  1622 raft_consensus.cc:359] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.832847  1622 raft_consensus.cc:385] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.833287  1622 raft_consensus.cc:740] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.832442  1619 ts_tablet_manager.cc:1434] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.165s	user 0.010s	sys 0.003s
I20260709 12:21:53.832617  1747 raft_consensus.cc:697] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:53.842893  1619 tablet_bootstrap.cc:492] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:53.842571  1747 consensus_queue.cc:237] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:53.836187  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 77651.4us
I20260709 12:21:53.846385  1783 consensus_queue.cc:1048] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:53.846841  1632 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.847530  1632 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.850983  1622 consensus_queue.cc:260] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.851636  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:53.851894  1632 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:53.853847  1622 ts_tablet_manager.cc:1434] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.001s
I20260709 12:21:53.855638  1622 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:21:53.836537  1463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 282862us
I20260709 12:21:53.857089  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.858124  1498 raft_consensus.cc:2468] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:53.858532  1388 leader_election.cc:304] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:53.859994  1857 raft_consensus.cc:3060] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.855333  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:53.827572  1623 raft_consensus.cc:740] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.867261  1623 consensus_queue.cc:260] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:53.869494  1623 ts_tablet_manager.cc:1434] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.318s	user 0.005s	sys 0.004s
I20260709 12:21:53.875977  1619 tablet_bootstrap.cc:654] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.878938  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.879119  1855 consensus_queue.cc:1048] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:53.880162  1537 leader_election.cc:304] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:53.881201  1855 raft_consensus.cc:2804] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.881652  1855 raft_consensus.cc:493] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.882068  1855 raft_consensus.cc:3060] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.881220  1496 raft_consensus.cc:2468] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 1.
I20260709 12:21:53.891712  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:53.896123  1847 raft_consensus.cc:493] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.896917  1847 raft_consensus.cc:515] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:53.899682  1847 leader_election.cc:290] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:53.904325  1857 raft_consensus.cc:2749] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:53.905903  1537 leader_election.cc:343] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.906107  1347 raft_consensus.cc:3060] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.915576  1623 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:53.927101  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a329febbdb4a35accb4f17719726dc" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:53.916596  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:53.943351  1622 tablet_bootstrap.cc:654] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.956745  1312 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:53.974807  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:53.986749  1855 raft_consensus.cc:515] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:53.989670  1855 leader_election.cc:290] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
W20260709 12:21:53.993532  1461 leader_election.cc:343] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:54.010677  1388 leader_election.cc:343] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:54.012560  1386 leader_election.cc:343] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.014705  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85193c93a364d41b3124bde25907dee" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:54.019968  1497 raft_consensus.cc:2468] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:54.020318  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:54.021198  1774 raft_consensus.cc:2804] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:54.020224  1498 raft_consensus.cc:3060] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.036424  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:54.037307  1495 raft_consensus.cc:3060] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.037623  1774 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 76217635dae3410b96baf380d386cbee)
I20260709 12:21:54.041643  1618 tablet_bootstrap.cc:654] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:54.037741  1537 leader_election.cc:343] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.038314  1774 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.059167  1537 leader_election.cc:304] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:54.017190  1386 leader_election.cc:304] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:54.067708  1856 raft_consensus.cc:2804] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.068392  1856 raft_consensus.cc:697] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:54.069674  1856 consensus_queue.cc:237] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.082705  1241 catalog_manager.cc:5697] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:54.087325  1857 raft_consensus.cc:2749] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.088238  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.091698  1241 catalog_manager.cc:5697] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.100883  1423 raft_consensus.cc:2468] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
W20260709 12:21:54.086530  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 73970.1us
I20260709 12:21:54.114910  1619 tablet_bootstrap.cc:492] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:54.115600  1619 ts_tablet_manager.cc:1403] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.273s	user 0.011s	sys 0.045s
I20260709 12:21:54.122834  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.141439  1312 leader_election.cc:304] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:54.142477  1850 raft_consensus.cc:2749] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.151058  1463 leader_election.cc:304] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:54.151672  1498 raft_consensus.cc:2468] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:54.152526  1796 raft_consensus.cc:2804] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.160792  1495 raft_consensus.cc:2468] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:54.161047  1796 raft_consensus.cc:697] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:54.152737  1622 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:54.162058  1622 ts_tablet_manager.cc:1403] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.307s	user 0.012s	sys 0.010s
I20260709 12:21:54.162340  1796 consensus_queue.cc:237] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.165272  1622 raft_consensus.cc:359] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.166036  1622 raft_consensus.cc:385] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.154624  1623 tablet_bootstrap.cc:654] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.166306  1622 raft_consensus.cc:740] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.178731  1670 raft_consensus.cc:493] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.179313  1670 raft_consensus.cc:515] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.183174  1670 leader_election.cc:290] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:54.185164  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:54.187296  1461 leader_election.cc:343] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.187916  1461 leader_election.cc:304] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:54.190002  1847 raft_consensus.cc:2749] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:54.186689  1535 leader_election.cc:343] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.182057  1622 consensus_queue.cc:260] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.195362  1618 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:54.196231  1618 ts_tablet_manager.cc:1403] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.396s	user 0.014s	sys 0.004s
I20260709 12:21:54.209306  1618 raft_consensus.cc:359] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.210147  1618 raft_consensus.cc:385] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.210428  1618 raft_consensus.cc:740] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.211198  1618 consensus_queue.cc:260] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.217103  1622 ts_tablet_manager.cc:1434] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 12:21:54.228447  1619 raft_consensus.cc:359] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.230268  1619 raft_consensus.cc:385] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.230583  1619 raft_consensus.cc:740] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.214916  1537 leader_election.cc:304] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 95782952bdee421094cada4ea955015d
I20260709 12:21:54.233543  1618 ts_tablet_manager.cc:1434] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.001s
I20260709 12:21:54.234476  1856 raft_consensus.cc:2804] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.235515  1856 raft_consensus.cc:697] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:54.236656  1623 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:54.238885  1622 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:54.237279  1623 ts_tablet_manager.cc:1403] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.322s	user 0.008s	sys 0.008s
I20260709 12:21:54.238348  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:54.241904  1420 raft_consensus.cc:2468] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:54.236810  1856 consensus_queue.cc:237] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.237564  1619 consensus_queue.cc:260] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.247184  1537 leader_election.cc:304] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:54.248118  1618 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:54.248378  1670 raft_consensus.cc:2804] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.250123  1670 raft_consensus.cc:493] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.250697  1670 raft_consensus.cc:3060] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.248755  1623 raft_consensus.cc:359] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.254787  1623 raft_consensus.cc:385] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.255193  1623 raft_consensus.cc:740] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.260404  1623 consensus_queue.cc:260] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.263806  1347 raft_consensus.cc:3060] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.268833  1774 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.270107  1348 raft_consensus.cc:1240] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Rejecting Update request from peer 76217635dae3410b96baf380d386cbee for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:54.273633  1857 consensus_queue.cc:1048] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:54.276525  1857 consensus_queue.cc:1048] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:54.279245  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:54.283030  1619 ts_tablet_manager.cc:1434] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.167s	user 0.001s	sys 0.009s
I20260709 12:21:54.293694  1619 tablet_bootstrap.cc:492] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:54.290547  1856 consensus_queue.cc:1059] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:21:54.305117  1856 raft_consensus.cc:3055] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:54.305539  1856 raft_consensus.cc:740] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:54.305351  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:54.308473  1623 ts_tablet_manager.cc:1434] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.001s
I20260709 12:21:54.308573  1857 consensus_queue.cc:1048] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.310447  1774 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:54.310875  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:54.311672  1618 tablet_bootstrap.cc:654] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.313273  1622 tablet_bootstrap.cc:654] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.312400  1623 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:54.307475  1856 consensus_queue.cc:260] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.321528  1847 raft_consensus.cc:493] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.326869  1847 raft_consensus.cc:515] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.329615  1847 leader_election.cc:290] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:54.330410  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:54.331532  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
W20260709 12:21:54.332195  1461 leader_election.cc:343] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.313052  1670 raft_consensus.cc:515] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.332836  1856 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.332619  1420 raft_consensus.cc:2468] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:54.350093  1670 leader_election.cc:290] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:54.352993  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:54.353871  1463 leader_election.cc:304] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
W20260709 12:21:54.354635  1535 leader_election.cc:343] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.356033  1796 raft_consensus.cc:2804] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.356657  1796 raft_consensus.cc:493] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.357131  1796 raft_consensus.cc:3060] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.359716  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:54.360415  1423 raft_consensus.cc:3060] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.365335  1623 tablet_bootstrap.cc:654] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.365969  1619 tablet_bootstrap.cc:654] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.373387  1796 raft_consensus.cc:515] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.376024  1796 leader_election.cc:290] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:54.376066  1857 consensus_queue.cc:1048] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:54.382006  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:54.384294  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:54.384562  1571 raft_consensus.cc:2468] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:21:54.385066  1420 raft_consensus.cc:3060] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:54.386886  1461 leader_election.cc:343] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.390553  1423 raft_consensus.cc:2468] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:54.391916  1537 leader_election.cc:304] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:54.395426  1670 raft_consensus.cc:2804] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.395629  1622 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:54.396024  1670 raft_consensus.cc:697] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:54.396332  1622 ts_tablet_manager.cc:1403] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.158s	user 0.013s	sys 0.009s
I20260709 12:21:54.397086  1670 consensus_queue.cc:237] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.399821  1622 raft_consensus.cc:359] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.402387  1618 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:54.400629  1622 raft_consensus.cc:385] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.406535  1622 raft_consensus.cc:740] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.406719  1618 ts_tablet_manager.cc:1403] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.166s	user 0.011s	sys 0.010s
I20260709 12:21:54.395699  1420 raft_consensus.cc:2468] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:54.417872  1463 leader_election.cc:304] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:54.418707  1622 consensus_queue.cc:260] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.421276  1622 ts_tablet_manager.cc:1434] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:21:54.422631  1622 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:54.427816  1618 raft_consensus.cc:359] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.428565  1618 raft_consensus.cc:385] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.428844  1618 raft_consensus.cc:740] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.429600  1618 consensus_queue.cc:260] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.430158  1619 tablet_bootstrap.cc:492] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:54.430778  1619 ts_tablet_manager.cc:1403] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.137s	user 0.008s	sys 0.010s
I20260709 12:21:54.432660  1618 ts_tablet_manager.cc:1434] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:21:54.430632  1847 raft_consensus.cc:2804] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.434175  1618 tablet_bootstrap.cc:492] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:54.434092  1847 raft_consensus.cc:697] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:54.435523  1847 consensus_queue.cc:237] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.440267  1619 raft_consensus.cc:359] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.446204  1619 raft_consensus.cc:385] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.446622  1619 raft_consensus.cc:740] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.447755  1619 consensus_queue.cc:260] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.462812  1619 ts_tablet_manager.cc:1434] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 12:21:54.463979  1619 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:54.467576  1310 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:54.469812  1774 raft_consensus.cc:2804] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:54.471268  1774 raft_consensus.cc:697] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:21:54.472422  1774 consensus_queue.cc:237] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.475802  1857 raft_consensus.cc:493] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.476394  1857 raft_consensus.cc:515] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.478615  1618 tablet_bootstrap.cc:654] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.479591  1857 leader_election.cc:290] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:54.480607  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:54.481472  1623 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:54.482007  1623 ts_tablet_manager.cc:1403] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.170s	user 0.011s	sys 0.016s
I20260709 12:21:54.482654  1850 raft_consensus.cc:493] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.480754  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.483259  1850 raft_consensus.cc:515] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.484920  1623 raft_consensus.cc:359] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.485689  1623 raft_consensus.cc:385] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.487672  1619 tablet_bootstrap.cc:654] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:54.482678  1388 leader_election.cc:343] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:54.489466  1386 leader_election.cc:343] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.489472  1622 tablet_bootstrap.cc:654] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.489604  1850 leader_election.cc:290] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:54.485950  1623 raft_consensus.cc:740] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.492764  1623 consensus_queue.cc:260] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.495342  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.497080  1386 leader_election.cc:304] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:54.498153  1857 raft_consensus.cc:2749] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.500501  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:54.501619  1420 raft_consensus.cc:2393] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
I20260709 12:21:54.502748  1619 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:54.503263  1619 ts_tablet_manager.cc:1403] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 12:21:54.503027  1312 leader_election.cc:304] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d
I20260709 12:21:54.506237  1619 raft_consensus.cc:359] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.506974  1619 raft_consensus.cc:385] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.507241  1619 raft_consensus.cc:740] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.507956  1619 consensus_queue.cc:260] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.511763  1774 raft_consensus.cc:3060] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.512025  1619 ts_tablet_manager.cc:1434] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:54.511453  1241 catalog_manager.cc:5697] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.513309  1241 catalog_manager.cc:5697] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.513327  1619 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:54.516573  1241 catalog_manager.cc:5697] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:54.519054  1623 ts_tablet_manager.cc:1434] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 12:21:54.520402  1623 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:54.525097  1618 tablet_bootstrap.cc:492] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:54.525758  1618 ts_tablet_manager.cc:1403] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.002s
I20260709 12:21:54.529543  1619 tablet_bootstrap.cc:654] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.529345  1618 raft_consensus.cc:359] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.529985  1618 raft_consensus.cc:385] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.530254  1618 raft_consensus.cc:740] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.531152  1618 consensus_queue.cc:260] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.535123  1618 ts_tablet_manager.cc:1434] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:21:54.539415  1619 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:54.539875  1619 ts_tablet_manager.cc:1403] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.003s
I20260709 12:21:54.543637  1240 catalog_manager.cc:5697] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:54.544407  1619 raft_consensus.cc:359] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.545038  1619 raft_consensus.cc:385] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.545347  1240 catalog_manager.cc:5697] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:54.546670  1347 raft_consensus.cc:3060] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.549497  1622 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:54.549981  1622 ts_tablet_manager.cc:1403] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.128s	user 0.010s	sys 0.018s
I20260709 12:21:54.551966  1619 raft_consensus.cc:740] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.552755  1619 consensus_queue.cc:260] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.554832  1619 ts_tablet_manager.cc:1434] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:21:54.556149  1622 raft_consensus.cc:359] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.556821  1622 raft_consensus.cc:385] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.557112  1622 raft_consensus.cc:740] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.557924  1622 consensus_queue.cc:260] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:21:54.558537  1461 consensus_peers.cc:597] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:54.559463  1622 ts_tablet_manager.cc:1434] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:21:54.560544  1622 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:54.561285  1774 raft_consensus.cc:2749] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.562026  1670 consensus_queue.cc:1048] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.562077  1619 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:54.565424  1747 raft_consensus.cc:493] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.566038  1747 raft_consensus.cc:515] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.569667  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:54.570212  1619 tablet_bootstrap.cc:654] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.570705  1623 tablet_bootstrap.cc:654] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.570899  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:21:54.572983  1386 leader_election.cc:343] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:54.574751  1388 leader_election.cc:343] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.575289  1388 leader_election.cc:304] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:54.576305  1857 raft_consensus.cc:2749] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.577301  1622 tablet_bootstrap.cc:654] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.579098  1618 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:54.579568  1747 leader_election.cc:290] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:54.581516  1783 raft_consensus.cc:493] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.583463  1619 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:54.583273  1783 raft_consensus.cc:515] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.583972  1619 ts_tablet_manager.cc:1403] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 12:21:54.585664  1618 tablet_bootstrap.cc:654] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.585933  1783 leader_election.cc:290] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.588892  1623 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:54.589392  1623 ts_tablet_manager.cc:1403] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.069s	user 0.002s	sys 0.008s
I20260709 12:21:54.591949  1622 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:54.591779  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:54.592459  1622 ts_tablet_manager.cc:1403] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.032s	user 0.000s	sys 0.011s
I20260709 12:21:54.592305  1623 raft_consensus.cc:359] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.592633  1496 raft_consensus.cc:2468] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:54.593004  1623 raft_consensus.cc:385] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.593387  1623 raft_consensus.cc:740] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.595664  1622 raft_consensus.cc:359] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.595944  1623 consensus_queue.cc:260] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.590070  1619 raft_consensus.cc:359] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.596949  1619 raft_consensus.cc:385] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.597237  1619 raft_consensus.cc:740] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.597596  1537 leader_election.cc:304] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:54.597863  1619 consensus_queue.cc:260] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.596344  1622 raft_consensus.cc:385] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.598869  1622 raft_consensus.cc:740] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.599690  1622 consensus_queue.cc:260] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.600390  1623 ts_tablet_manager.cc:1434] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:21:54.600983  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:54.601691  1623 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:54.601841  1622 ts_tablet_manager.cc:1434] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:21:54.605504  1619 ts_tablet_manager.cc:1434] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
W20260709 12:21:54.605868  1535 leader_election.cc:343] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.606950  1619 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:54.607854  1856 raft_consensus.cc:2804] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.608402  1856 raft_consensus.cc:493] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.608804  1856 raft_consensus.cc:3060] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.610046  1622 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:54.612098  1623 tablet_bootstrap.cc:654] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.619231  1619 tablet_bootstrap.cc:654] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.627488  1856 raft_consensus.cc:515] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.628742  1244 catalog_manager.cc:5697] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:54.642074  1618 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:54.642653  1618 ts_tablet_manager.cc:1403] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.005s
I20260709 12:21:54.647886  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:54.649711  1496 raft_consensus.cc:3060] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.649175  1423 raft_consensus.cc:3060] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.649737  1618 raft_consensus.cc:359] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.657419  1618 raft_consensus.cc:385] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.657925  1618 raft_consensus.cc:740] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.667338  1622 tablet_bootstrap.cc:654] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.665578  1850 raft_consensus.cc:493] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.667945  1618 consensus_queue.cc:260] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.668368  1850 raft_consensus.cc:515] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.671867  1856 leader_election.cc:290] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.667135  1855 consensus_queue.cc:1048] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:21:54.674155  1850 leader_election.cc:290] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:54.676223  1619 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:54.676739  1619 ts_tablet_manager.cc:1403] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.007s
I20260709 12:21:54.676996  1418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:54.677639  1747 raft_consensus.cc:493] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.677901  1418 raft_consensus.cc:2393] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:54.687057  1747 raft_consensus.cc:515] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.676517  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68fc83ac5034733b526df7c6ffef4f8" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:54.689738  1670 raft_consensus.cc:493] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.670684  1618 ts_tablet_manager.cc:1434] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.003s
I20260709 12:21:54.701007  1670 raft_consensus.cc:515] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.703671  1618 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:54.711064  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:54.714826  1312 leader_election.cc:304] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:54.716104  1850 raft_consensus.cc:3060] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.719696  1670 leader_election.cc:290] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:54.720570  1857 raft_consensus.cc:493] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.724879  1857 raft_consensus.cc:515] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:54.725610  1623 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:54.726195  1623 ts_tablet_manager.cc:1403] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.125s	user 0.034s	sys 0.012s
W20260709 12:21:54.721853  1535 leader_election.cc:343] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.723373  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012be77ba30c48409e5b4e286fa90432" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:54.729492  1535 leader_election.cc:343] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.731840  1857 leader_election.cc:290] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:54.733402  1850 raft_consensus.cc:2749] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.733177  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:54.735073  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012be77ba30c48409e5b4e286fa90432" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
W20260709 12:21:54.735213  1388 leader_election.cc:343] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.735628  1619 raft_consensus.cc:359] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.735951  1571 raft_consensus.cc:2468] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:54.736429  1619 raft_consensus.cc:385] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.736783  1619 raft_consensus.cc:740] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.737553  1619 consensus_queue.cc:260] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.737879  1386 leader_election.cc:304] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:54.733152  1623 raft_consensus.cc:359] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.740829  1623 raft_consensus.cc:385] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.741472  1623 raft_consensus.cc:740] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
W20260709 12:21:54.711071  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 55003.8us
I20260709 12:21:54.744228  1846 raft_consensus.cc:493] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.754689  1846 raft_consensus.cc:515] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.755113  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012be77ba30c48409e5b4e286fa90432" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:54.756527  1386 leader_election.cc:343] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.757206  1846 leader_election.cc:290] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.745097  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:54.759625  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:54.746618  1847 raft_consensus.cc:493] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:54.761045  1461 leader_election.cc:343] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.748503  1797 consensus_queue.cc:1048] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.761932  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.753520  1702 raft_consensus.cc:493] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.763276  1702 raft_consensus.cc:515] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.764825  1633 consensus_queue.cc:1048] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.761260  1847 raft_consensus.cc:515] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
W20260709 12:21:54.770295  1461 leader_election.cc:343] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.770980  1461 leader_election.cc:304] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:54.772281  1623 consensus_queue.cc:260] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.774153  1847 leader_election.cc:290] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.775056  1623 ts_tablet_manager.cc:1434] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.049s	user 0.011s	sys 0.000s
I20260709 12:21:54.775617  1702 leader_election.cc:290] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:54.753942  1747 leader_election.cc:290] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.777637  1797 raft_consensus.cc:2749] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:54.779313  1386 leader_election.cc:343] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.779963  1386 leader_election.cc:304] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:54.782291  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.780164  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:54.757685  1619 ts_tablet_manager.cc:1434] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.002s
I20260709 12:21:54.744712  1746 raft_consensus.cc:2804] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.796357  1746 raft_consensus.cc:493] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.801596  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "454ec9762c6f480bbc5587abbce5f8f6" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.804920  1746 raft_consensus.cc:3060] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.805173  1571 raft_consensus.cc:2468] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:54.786123  1497 raft_consensus.cc:2468] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:54.808246  1388 leader_election.cc:304] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:54.797084  1619 tablet_bootstrap.cc:492] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:54.782938  1670 consensus_queue.cc:1048] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:21:54.786420  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "454ec9762c6f480bbc5587abbce5f8f6" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:54.782680  1747 raft_consensus.cc:2749] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:54.818440  1461 leader_election.cc:343] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.818902  1746 raft_consensus.cc:515] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.819680  1461 leader_election.cc:304] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:54.820911  1797 raft_consensus.cc:2804] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.821310  1797 raft_consensus.cc:493] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.821724  1797 raft_consensus.cc:3060] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.788447  1623 tablet_bootstrap.cc:492] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:54.825855  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012be77ba30c48409e5b4e286fa90432" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:54.826664  1571 raft_consensus.cc:3060] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.831699  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012be77ba30c48409e5b4e286fa90432" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:54.832981  1857 raft_consensus.cc:493] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:54.833359  1386 leader_election.cc:343] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.799666  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "012be77ba30c48409e5b4e286fa90432" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:54.834580  1423 raft_consensus.cc:2393] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76217635dae3410b96baf380d386cbee in current term 1: Already voted for candidate 95782952bdee421094cada4ea955015d in this term.
I20260709 12:21:54.837620  1857 raft_consensus.cc:515] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:54.838924  1618 tablet_bootstrap.cc:654] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.840111  1537 leader_election.cc:304] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:54.840889  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.841609  1623 tablet_bootstrap.cc:654] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.841671  1572 raft_consensus.cc:2468] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:54.844034  1386 leader_election.cc:304] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:54.846151  1619 tablet_bootstrap.cc:654] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.782454  1496 raft_consensus.cc:2468] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:54.856501  1847 raft_consensus.cc:493] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.864738  1847 raft_consensus.cc:515] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.846853  1747 raft_consensus.cc:2804] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.866750  1747 raft_consensus.cc:493] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.867189  1747 raft_consensus.cc:3060] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.871089  1702 raft_consensus.cc:2804] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:54.873833  1847 leader_election.cc:290] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:54.869993  1537 leader_election.cc:304] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:54.821290  1746 leader_election.cc:290] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.879221  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:54.881584  1847 consensus_queue.cc:1048] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:54.885428  1901 raft_consensus.cc:2804] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.885125  1797 raft_consensus.cc:515] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.886006  1622 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:54.886879  1622 ts_tablet_manager.cc:1403] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.277s	user 0.016s	sys 0.019s
I20260709 12:21:54.887780  1797 leader_election.cc:290] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.886014  1901 raft_consensus.cc:697] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
W20260709 12:21:54.890131  1461 leader_election.cc:343] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.892138  1571 raft_consensus.cc:2468] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:54.891661  1901 consensus_queue.cc:237] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.892840  1670 raft_consensus.cc:2749] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.893250  1857 leader_election.cc:290] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:54.897794  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:54.898842  1423 raft_consensus.cc:2468] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:54.905889  1463 leader_election.cc:304] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:54.909124  1847 raft_consensus.cc:2804] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.909631  1847 raft_consensus.cc:493] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.910017  1847 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.911669  1497 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.905728  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "454ec9762c6f480bbc5587abbce5f8f6" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:54.897032  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "454ec9762c6f480bbc5587abbce5f8f6" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:54.913630  1572 raft_consensus.cc:3060] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:54.913615  1461 leader_election.cc:343] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.919631  1702 raft_consensus.cc:493] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 76217635dae3410b96baf380d386cbee)
I20260709 12:21:54.920547  1497 raft_consensus.cc:1275] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:54.925858  1702 raft_consensus.cc:3060] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:54.927000  1622 raft_consensus.cc:359] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.922569  1670 consensus_queue.cc:1048] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.927771  1622 raft_consensus.cc:385] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.928155  1622 raft_consensus.cc:740] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.927456  1850 consensus_queue.cc:1048] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:54.928910  1622 consensus_queue.cc:260] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:54.948120  1622 ts_tablet_manager.cc:1434] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 12:21:54.949671  1622 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:54.968263  1847 raft_consensus.cc:515] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:54.968725  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:54.980923  1244 catalog_manager.cc:5697] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.995644  1847 leader_election.cc:290] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:55.005800  1623 tablet_bootstrap.cc:492] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:55.006405  1623 ts_tablet_manager.cc:1403] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.218s	user 0.009s	sys 0.048s
I20260709 12:21:55.008771  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:55.009079  1670 consensus_queue.cc:1048] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.009291  1702 raft_consensus.cc:515] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.010454  1423 raft_consensus.cc:1240] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Rejecting Update request from peer 76217635dae3410b96baf380d386cbee for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.009080  1747 raft_consensus.cc:515] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
W20260709 12:21:55.016137  1461 leader_election.cc:343] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.017766  1702 leader_election.cc:290] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:54.990069  1386 leader_election.cc:304] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:55.036288  1623 raft_consensus.cc:359] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.037194  1623 raft_consensus.cc:385] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.037498  1623 raft_consensus.cc:740] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.039347  1902 consensus_queue.cc:1059] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.040479  1902 raft_consensus.cc:3055] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.040926  1902 raft_consensus.cc:740] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:55.041745  1572 raft_consensus.cc:2468] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:55.041934  1902 consensus_queue.cc:260] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.038235  1623 consensus_queue.cc:260] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.054419  1623 ts_tablet_manager.cc:1434] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 12:21:55.052485  1746 raft_consensus.cc:2804] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.059021  1461 leader_election.cc:304] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:55.054080  1902 raft_consensus.cc:3060] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.057130  1571 raft_consensus.cc:1275] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.070758  1347 raft_consensus.cc:3060] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.063117  1847 raft_consensus.cc:2804] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.074082  1847 raft_consensus.cc:697] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:55.075030  1847 consensus_queue.cc:237] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.077961  1747 leader_election.cc:290] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:55.080468  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:55.081092  1423 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.085124  1622 tablet_bootstrap.cc:654] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.057948  1746 raft_consensus.cc:697] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
W20260709 12:21:55.056608  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 89691.6us
I20260709 12:21:55.090005  1623 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:55.095022  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:55.098815  1866 raft_consensus.cc:493] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.099392  1866 raft_consensus.cc:515] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.107349  1916 consensus_queue.cc:1048] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.060374  1913 raft_consensus.cc:493] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.113734  1746 consensus_queue.cc:237] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:55.119073  1850 consensus_queue.cc:1048] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.109455  1423 raft_consensus.cc:2468] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:55.114399  1866 leader_election.cc:290] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:55.134954  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:55.141149  1619 tablet_bootstrap.cc:492] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:55.141974  1619 ts_tablet_manager.cc:1403] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.345s	user 0.017s	sys 0.013s
I20260709 12:21:55.143913  1618 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:55.144531  1618 ts_tablet_manager.cc:1403] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.441s	user 0.027s	sys 0.014s
I20260709 12:21:55.158147  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:55.156100  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:55.160624  1572 raft_consensus.cc:3060] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.117425  1913 raft_consensus.cc:515] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.166525  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59009719f3a54ceebc3c12b1687b9000" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:55.193580  1463 leader_election.cc:304] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:55.195518  1914 raft_consensus.cc:2804] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.196120  1914 raft_consensus.cc:697] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:55.197049  1914 consensus_queue.cc:237] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.199663  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:55.200608  1498 raft_consensus.cc:2393] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
W20260709 12:21:55.180585  1386 leader_election.cc:343] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.201009  1623 tablet_bootstrap.cc:654] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.215049  1245 catalog_manager.cc:5697] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:55.228960  1572 raft_consensus.cc:2468] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:55.235225  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:55.240083  1310 leader_election.cc:304] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:55.241843  1569 raft_consensus.cc:2468] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:21:55.242697  1866 raft_consensus.cc:3060] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:55.243711  1921 log.cc:927] Time spent T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:21:55.235695  1619 raft_consensus.cc:359] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:55.224524  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 52278.5us
I20260709 12:21:55.246662  1619 raft_consensus.cc:385] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.246994  1619 raft_consensus.cc:740] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
W20260709 12:21:55.248397  1386 leader_election.cc:343] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.248953  1619 consensus_queue.cc:260] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.251144  1619 ts_tablet_manager.cc:1434] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.002s
I20260709 12:21:55.252238  1619 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
W20260709 12:21:55.255438  1920 log.cc:927] Time spent T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:21:55.265151  1618 raft_consensus.cc:359] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.265654  1853 raft_consensus.cc:493] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.266065  1618 raft_consensus.cc:385] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.266356  1853 raft_consensus.cc:515] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.267428  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beebaa55c4f41b1b2a0bb635654d53a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:55.266528  1618 raft_consensus.cc:740] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.276348  1618 consensus_queue.cc:260] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.279681  1913 leader_election.cc:290] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:55.280524  1853 leader_election.cc:290] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:55.281635  1902 raft_consensus.cc:493] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.282305  1902 raft_consensus.cc:515] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.285071  1902 leader_election.cc:290] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:55.287322  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:55.290337  1622 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:55.290954  1622 ts_tablet_manager.cc:1403] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.342s	user 0.023s	sys 0.007s
I20260709 12:21:55.290659  1241 catalog_manager.cc:5697] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.292938  1866 raft_consensus.cc:2749] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.290675  1618 ts_tablet_manager.cc:1434] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.146s	user 0.005s	sys 0.005s
I20260709 12:21:55.293442  1622 raft_consensus.cc:359] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.294256  1622 raft_consensus.cc:385] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.294587  1622 raft_consensus.cc:740] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.294730  1618 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:55.295398  1622 consensus_queue.cc:260] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.297565  1622 ts_tablet_manager.cc:1434] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:55.299952  1622 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:55.309491  1619 tablet_bootstrap.cc:654] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.310398  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:55.312095  1423 raft_consensus.cc:2393] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76217635dae3410b96baf380d386cbee in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:55.311940  1386 leader_election.cc:304] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:55.312920  1917 raft_consensus.cc:2804] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.313388  1917 raft_consensus.cc:697] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:55.314203  1917 consensus_queue.cc:237] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.317567  1537 leader_election.cc:304] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:55.319393  1622 tablet_bootstrap.cc:654] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.319917  1670 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.343547  1240 catalog_manager.cc:5697] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 2 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.347677  1623 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:55.347399  1386 leader_election.cc:304] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:55.348354  1623 ts_tablet_manager.cc:1403] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.260s	user 0.012s	sys 0.006s
I20260709 12:21:55.351287  1917 raft_consensus.cc:2804] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.351852  1917 raft_consensus.cc:697] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:55.355468  1312 leader_election.cc:304] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:55.354657  1623 raft_consensus.cc:359] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.357044  1623 raft_consensus.cc:385] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.357321  1623 raft_consensus.cc:740] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.352721  1917 consensus_queue.cc:237] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:55.364941  1623 consensus_queue.cc:260] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.372557  1670 raft_consensus.cc:2749] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.362270  1866 raft_consensus.cc:2749] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:55.381561  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:21:55.386025  1388 leader_election.cc:343] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.389151  1622 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:55.388200  1241 catalog_manager.cc:5697] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.389725  1622 ts_tablet_manager.cc:1403] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.091s	user 0.018s	sys 0.019s
I20260709 12:21:55.390915  1623 ts_tablet_manager.cc:1434] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.002s
I20260709 12:21:55.396423  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:55.398152  1623 tablet_bootstrap.cc:492] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:55.411784  1618 tablet_bootstrap.cc:654] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.416127  1670 consensus_queue.cc:1048] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.427476  1902 consensus_queue.cc:1048] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.438970  1241 catalog_manager.cc:5697] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:55.448503  1386 leader_election.cc:343] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.451594  1386 leader_election.cc:304] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:55.452405  1917 raft_consensus.cc:2749] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.454290  1622 raft_consensus.cc:359] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:55.458153  1622 raft_consensus.cc:385] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.457650  1623 tablet_bootstrap.cc:654] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.460976  1622 raft_consensus.cc:740] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.464800  1622 consensus_queue.cc:260] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:55.479773  1619 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:55.480547  1619 ts_tablet_manager.cc:1403] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.229s	user 0.033s	sys 0.024s
I20260709 12:21:55.484290  1619 raft_consensus.cc:359] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:55.484997  1619 raft_consensus.cc:385] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.485278  1619 raft_consensus.cc:740] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.486126  1619 consensus_queue.cc:260] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:55.488655  1619 ts_tablet_manager.cc:1434] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:55.490106  1619 tablet_bootstrap.cc:492] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:55.475299  1797 raft_consensus.cc:493] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.491318  1797 raft_consensus.cc:515] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.493614  1797 leader_election.cc:290] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:55.495085  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:55.495814  1423 raft_consensus.cc:2468] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:55.498080  1463 leader_election.cc:304] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:55.502573  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:55.504979  1461 leader_election.cc:343] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.513336  1797 raft_consensus.cc:2804] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.513968  1797 raft_consensus.cc:493] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.514412  1797 raft_consensus.cc:3060] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:55.548061  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:55.551695  1619 tablet_bootstrap.cc:654] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.556828  1622 ts_tablet_manager.cc:1434] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.166s	user 0.009s	sys 0.000s
I20260709 12:21:55.558077  1622 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:55.574869  1347 raft_consensus.cc:3060] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.590294  1916 consensus_queue.cc:1048] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.598537  1623 tablet_bootstrap.cc:492] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:55.599092  1623 ts_tablet_manager.cc:1403] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.201s	user 0.037s	sys 0.037s
I20260709 12:21:55.598881  1917 consensus_queue.cc:1048] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.604472  1623 raft_consensus.cc:359] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.605202  1623 raft_consensus.cc:385] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.605469  1623 raft_consensus.cc:740] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.607187  1618 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:55.607811  1618 ts_tablet_manager.cc:1403] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.313s	user 0.042s	sys 0.019s
I20260709 12:21:55.609912  1797 raft_consensus.cc:515] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.635262  1622 tablet_bootstrap.cc:654] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.637039  1797 leader_election.cc:290] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:55.638187  1418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:55.638978  1418 raft_consensus.cc:3060] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.638849  1917 consensus_queue.cc:1048] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:55.641716  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:55.622390  1619 tablet_bootstrap.cc:492] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:55.619727  1618 raft_consensus.cc:359] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.647794  1619 ts_tablet_manager.cc:1403] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.158s	user 0.011s	sys 0.003s
W20260709 12:21:55.649194  1461 leader_election.cc:343] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.647817  1618 raft_consensus.cc:385] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.652050  1618 raft_consensus.cc:740] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.652863  1618 consensus_queue.cc:260] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.624382  1346 raft_consensus.cc:3060] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.657517  1619 raft_consensus.cc:359] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.611004  1623 consensus_queue.cc:260] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.658356  1619 raft_consensus.cc:385] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.658752  1619 raft_consensus.cc:740] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.667500  1618 ts_tablet_manager.cc:1434] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.001s
I20260709 12:21:55.669611  1623 ts_tablet_manager.cc:1434] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 12:21:55.671337  1846 consensus_queue.cc:1048] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:55.673717  1623 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:55.674770  1670 raft_consensus.cc:493] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.675257  1670 raft_consensus.cc:515] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.677206  1670 leader_election.cc:290] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:55.677366  1746 raft_consensus.cc:493] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.677865  1746 raft_consensus.cc:515] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:55.678648  1914 consensus_queue.cc:1048] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.680475  1746 leader_election.cc:290] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:55.680845  1914 consensus_queue.cc:1048] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.683429  1619 consensus_queue.cc:260] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.685585  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1ad92910454a39a4ebb1ea41c3c376" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:55.688123  1418 raft_consensus.cc:2468] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:55.690392  1463 leader_election.cc:304] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:55.692279  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1ad92910454a39a4ebb1ea41c3c376" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:55.692837  1846 raft_consensus.cc:2804] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.688277  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:55.694361  1498 raft_consensus.cc:2468] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:55.692940  1618 tablet_bootstrap.cc:492] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:55.692351  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:55.697510  1535 leader_election.cc:343] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:55.688525  1386 leader_election.cc:343] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.698642  1902 raft_consensus.cc:493] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.699499  1902 raft_consensus.cc:515] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:55.698732  1386 leader_election.cc:343] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.702627  1386 leader_election.cc:304] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:55.704430  1418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:55.705230  1418 raft_consensus.cc:2468] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:55.706697  1537 leader_election.cc:304] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:55.707612  1619 ts_tablet_manager.cc:1434] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 12:21:55.707700  1670 raft_consensus.cc:2804] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.708254  1670 raft_consensus.cc:493] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.708618  1670 raft_consensus.cc:3060] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.708786  1623 tablet_bootstrap.cc:654] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.702231  1902 leader_election.cc:290] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:55.711894  1571 raft_consensus.cc:1275] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.714264  1746 raft_consensus.cc:2749] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.715138  1746 consensus_queue.cc:1048] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:55.716313  1537 leader_election.cc:304] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:55.717876  1902 raft_consensus.cc:2804] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.718324  1902 raft_consensus.cc:493] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.720089  1846 raft_consensus.cc:697] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:55.718752  1902 raft_consensus.cc:3060] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.721215  1846 consensus_queue.cc:237] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.708814  1619 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:55.725780  1498 raft_consensus.cc:3060] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.729144  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:55.742741  1796 raft_consensus.cc:493] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.736932  1498 raft_consensus.cc:1275] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.743414  1796 raft_consensus.cc:515] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:21:55.742111  1537 leader_election.cc:343] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.746748  1796 leader_election.cc:290] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:55.750229  1418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:55.751008  1418 raft_consensus.cc:2468] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:55.752394  1622 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:55.752327  1463 leader_election.cc:304] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:55.752991  1747 consensus_queue.cc:1048] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.753443  1619 tablet_bootstrap.cc:654] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.753363  1796 raft_consensus.cc:2804] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.754050  1796 raft_consensus.cc:493] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.754428  1796 raft_consensus.cc:3060] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.757522  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:55.764240  1461 leader_election.cc:343] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.768227  1622 ts_tablet_manager.cc:1403] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.210s	user 0.015s	sys 0.010s
I20260709 12:21:55.779299  1618 tablet_bootstrap.cc:654] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.782728  1670 raft_consensus.cc:515] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:55.795805  1670 leader_election.cc:290] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:55.797560  1902 raft_consensus.cc:515] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.800092  1902 leader_election.cc:290] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:55.800927  1747 raft_consensus.cc:493] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.801301  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:55.801846  1747 raft_consensus.cc:515] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:55.803257  1535 leader_election.cc:343] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.805112  1747 leader_election.cc:290] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:55.807852  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:55.808368  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:55.808584  1423 raft_consensus.cc:3060] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.809125  1496 raft_consensus.cc:3060] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.818219  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:55.834656  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:55.835422  1241 catalog_manager.cc:5697] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:55.836859  1623 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:55.837575  1623 ts_tablet_manager.cc:1403] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.164s	user 0.028s	sys 0.017s
W20260709 12:21:55.861852  1386 leader_election.cc:343] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:55.867702  1386 consensus_peers.cc:597] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.868182  1496 raft_consensus.cc:2468] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:55.868911  1796 raft_consensus.cc:515] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:21:55.870589  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 43593.2us
W20260709 12:21:55.871500  1537 leader_election.cc:343] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.872260  1537 leader_election.cc:304] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:55.873040  1796 leader_election.cc:290] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:55.873342  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:55.875038  1670 raft_consensus.cc:2804] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.875553  1670 raft_consensus.cc:697] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:55.875880  1623 raft_consensus.cc:359] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.876294  1670 consensus_queue.cc:237] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.876752  1623 raft_consensus.cc:385] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:55.878737  1461 leader_election.cc:343] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.890071  1423 raft_consensus.cc:2468] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:55.878000  1623 raft_consensus.cc:740] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.893923  1623 consensus_queue.cc:260] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:55.900298  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:55.891459  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:55.904366  1420 raft_consensus.cc:3060] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.906201  1746 consensus_queue.cc:1048] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.909305  1537 leader_election.cc:304] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:55.910286  1633 raft_consensus.cc:2804] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.910815  1633 raft_consensus.cc:697] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:55.911706  1633 consensus_queue.cc:237] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:55.917469  1388 leader_election.cc:343] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.923367  1388 leader_election.cc:304] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:55.924193  1619 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:55.924687  1619 ts_tablet_manager.cc:1403] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.216s	user 0.015s	sys 0.026s
I20260709 12:21:55.927523  1619 raft_consensus.cc:359] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.928186  1619 raft_consensus.cc:385] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.928454  1619 raft_consensus.cc:740] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.922235  1622 raft_consensus.cc:359] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.930646  1622 raft_consensus.cc:385] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.930913  1622 raft_consensus.cc:740] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.931136  1917 raft_consensus.cc:2749] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.931583  1622 consensus_queue.cc:260] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.929128  1619 consensus_queue.cc:260] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.935732  1623 ts_tablet_manager.cc:1434] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.098s	user 0.011s	sys 0.000s
I20260709 12:21:55.936065  1619 ts_tablet_manager.cc:1434] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:55.936892  1623 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:55.945919  1619 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:55.961750  1420 raft_consensus.cc:2468] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:55.963363  1463 leader_election.cc:304] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:55.964607  1846 raft_consensus.cc:2804] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.965130  1846 raft_consensus.cc:697] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:55.966814  1618 tablet_bootstrap.cc:492] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:55.966255  1846 consensus_queue.cc:237] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:55.967628  1618 ts_tablet_manager.cc:1403] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.297s	user 0.035s	sys 0.075s
I20260709 12:21:55.974326  1622 ts_tablet_manager.cc:1434] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.206s	user 0.008s	sys 0.000s
W20260709 12:21:55.954548  1535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 112355us
I20260709 12:21:55.990635  1622 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:56.003971  1623 tablet_bootstrap.cc:654] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.024183  1619 tablet_bootstrap.cc:654] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.031688  1618 raft_consensus.cc:359] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.027158  1947 raft_consensus.cc:493] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.033221  1947 raft_consensus.cc:515] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.036069  1947 leader_election.cc:290] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:56.036700  1618 raft_consensus.cc:385] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.038041  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.038918  1618 raft_consensus.cc:740] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.041370  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:56.043402  1537 leader_election.cc:343] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.041581  1618 consensus_queue.cc:260] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.050954  1618 ts_tablet_manager.cc:1434] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.001s
I20260709 12:21:56.056234  1618 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
W20260709 12:21:56.052918  1535 leader_election.cc:343] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.052811  1241 catalog_manager.cc:5697] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:56.056435  1622 tablet_bootstrap.cc:654] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.058650  1535 leader_election.cc:304] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.047667  1244 catalog_manager.cc:5697] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.061594  1947 raft_consensus.cc:2749] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.063129  1244 catalog_manager.cc:5697] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:56.071058  1618 tablet_bootstrap.cc:654] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.171080  1619 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:56.171610  1619 ts_tablet_manager.cc:1403] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.235s	user 0.027s	sys 0.016s
I20260709 12:21:56.181411  1623 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:56.182340  1623 ts_tablet_manager.cc:1403] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.246s	user 0.021s	sys 0.004s
I20260709 12:21:56.208621  1623 raft_consensus.cc:359] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.209455  1623 raft_consensus.cc:385] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.209805  1623 raft_consensus.cc:740] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.210645  1623 consensus_queue.cc:260] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.212776  1796 consensus_queue.cc:1048] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.213827  1619 raft_consensus.cc:359] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.214567  1622 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:56.215384  1622 ts_tablet_manager.cc:1403] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.231s	user 0.020s	sys 0.034s
I20260709 12:21:56.217712  1623 ts_tablet_manager.cc:1434] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:21:56.218199  1622 raft_consensus.cc:359] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.219004  1622 raft_consensus.cc:385] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.219321  1622 raft_consensus.cc:740] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.214625  1619 raft_consensus.cc:385] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.219771  1619 raft_consensus.cc:740] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.220512  1623 tablet_bootstrap.cc:492] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:56.220152  1622 consensus_queue.cc:260] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.221325  1619 consensus_queue.cc:260] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.224509  1619 ts_tablet_manager.cc:1434] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.053s	user 0.002s	sys 0.005s
I20260709 12:21:56.226648  1622 ts_tablet_manager.cc:1434] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:56.230721  1618 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.230823  1622 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:56.231515  1618 ts_tablet_manager.cc:1403] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.176s	user 0.028s	sys 0.038s
I20260709 12:21:56.232784  1619 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:56.235363  1618 raft_consensus.cc:359] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.236088  1618 raft_consensus.cc:385] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.236353  1618 raft_consensus.cc:740] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.237061  1618 consensus_queue.cc:260] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.241477  1622 tablet_bootstrap.cc:654] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:56.243316  1461 consensus_peers.cc:597] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:56.245447  1618 ts_tablet_manager.cc:1434] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:21:56.246817  1618 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.254446  1618 tablet_bootstrap.cc:654] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.255446  1619 tablet_bootstrap.cc:654] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.266703  1622 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:56.267314  1622 ts_tablet_manager.cc:1403] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.008s
I20260709 12:21:56.274637  1622 raft_consensus.cc:359] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.275323  1622 raft_consensus.cc:385] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.275602  1622 raft_consensus.cc:740] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.276244  1622 consensus_queue.cc:260] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.284804  1623 tablet_bootstrap.cc:654] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.287168  1622 ts_tablet_manager.cc:1434] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:21:56.288249  1622 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:56.302510  1618 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.303025  1747 raft_consensus.cc:493] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.303653  1747 raft_consensus.cc:515] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.306092  1747 leader_election.cc:290] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:56.306756  1623 tablet_bootstrap.cc:492] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:56.307363  1623 ts_tablet_manager.cc:1403] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.087s	user 0.008s	sys 0.004s
I20260709 12:21:56.307847  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
W20260709 12:21:56.309300  1386 leader_election.cc:343] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.303198  1618 ts_tablet_manager.cc:1403] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.021s
I20260709 12:21:56.310431  1619 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:56.304276  1622 tablet_bootstrap.cc:654] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.311698  1619 ts_tablet_manager.cc:1403] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.079s	user 0.007s	sys 0.029s
I20260709 12:21:56.312585  1618 raft_consensus.cc:359] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.310123  1623 raft_consensus.cc:359] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.315174  1618 raft_consensus.cc:385] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.315538  1623 raft_consensus.cc:385] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.315758  1618 raft_consensus.cc:740] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.316968  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:56.316941  1618 consensus_queue.cc:260] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.314787  1619 raft_consensus.cc:359] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.319247  1619 raft_consensus.cc:385] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.319589  1618 ts_tablet_manager.cc:1434] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
W20260709 12:21:56.319586  1386 leader_election.cc:343] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.319633  1619 raft_consensus.cc:740] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.320657  1386 leader_election.cc:304] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:56.321079  1619 consensus_queue.cc:260] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.321834  1618 tablet_bootstrap.cc:492] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.322391  1746 raft_consensus.cc:2749] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.323474  1619 ts_tablet_manager.cc:1434] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:21:56.314708  1347 raft_consensus.cc:3060] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.324652  1619 tablet_bootstrap.cc:492] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:56.329259  1618 tablet_bootstrap.cc:654] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.319694  1623 raft_consensus.cc:740] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.331468  1623 consensus_queue.cc:260] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.334002  1623 ts_tablet_manager.cc:1434] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:21:56.335136  1623 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:56.341598  1618 tablet_bootstrap.cc:492] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.343044  1623 tablet_bootstrap.cc:654] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.346279  1619 tablet_bootstrap.cc:654] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.357754  1797 raft_consensus.cc:493] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.358472  1797 raft_consensus.cc:515] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.362221  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:56.363193  1747 raft_consensus.cc:493] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.363300  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:56.364924  1420 raft_consensus.cc:2393] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8246e0558124e98a58932ad7f4676e3 in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
I20260709 12:21:56.364512  1747 raft_consensus.cc:515] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.366225  1774 raft_consensus.cc:493] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.366350  1463 leader_election.cc:304] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d
I20260709 12:21:56.366961  1774 raft_consensus.cc:515] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.367471  1846 raft_consensus.cc:3060] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.368717  1619 tablet_bootstrap.cc:492] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:56.369257  1619 ts_tablet_manager.cc:1403] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.001s
I20260709 12:21:56.369912  1774 leader_election.cc:290] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:56.372555  1619 raft_consensus.cc:359] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.366410  1623 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:56.372975  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.385771  1623 ts_tablet_manager.cc:1403] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.007s
I20260709 12:21:56.389134  1623 raft_consensus.cc:359] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.390100  1623 raft_consensus.cc:385] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.390384  1623 raft_consensus.cc:740] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.390838  1846 raft_consensus.cc:2749] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.391254  1623 consensus_queue.cc:260] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.375563  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e9e93f05e547be9a7e726a1f4a419f" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:56.393020  1420 raft_consensus.cc:2393] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:56.395087  1947 consensus_queue.cc:1048] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:56.397393  1623 ts_tablet_manager.cc:1434] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:21:56.379050  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c312c9473d0e47e0816684df0c14a27f" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:56.384680  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c312c9473d0e47e0816684df0c14a27f" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.398573  1623 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:56.398834  1498 raft_consensus.cc:2468] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:56.399453  1853 raft_consensus.cc:493] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.374271  1670 consensus_queue.cc:1048] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.400081  1853 raft_consensus.cc:515] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.400995  1619 raft_consensus.cc:385] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.401325  1619 raft_consensus.cc:740] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.402086  1619 consensus_queue.cc:260] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.363780  1797 leader_election.cc:290] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:56.402416  1853 leader_election.cc:290] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:56.403276  1312 leader_election.cc:304] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.419266  1866 raft_consensus.cc:3060] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:56.423676  1535 consensus_peers.cc:597] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:56.424286  1535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 46535.4us
I20260709 12:21:56.436641  1747 leader_election.cc:290] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:56.440732  1846 consensus_queue.cc:1048] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.443208  1622 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:56.443604  1618 ts_tablet_manager.cc:1403] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.122s	user 0.011s	sys 0.007s
I20260709 12:21:56.443789  1622 ts_tablet_manager.cc:1403] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.156s	user 0.027s	sys 0.048s
I20260709 12:21:56.410431  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:56.416129  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:21:56.408911  1386 leader_election.cc:343] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:56.455071  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 46238.3us
W20260709 12:21:56.458164  1386 leader_election.cc:343] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.461897  1388 leader_election.cc:304] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.463790  1747 raft_consensus.cc:2804] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:56.463964  1388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 47889.5us
I20260709 12:21:56.464203  1747 raft_consensus.cc:493] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.465145  1747 raft_consensus.cc:3060] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.446662  1622 raft_consensus.cc:359] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.466475  1622 raft_consensus.cc:385] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.466782  1622 raft_consensus.cc:740] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.467461  1622 consensus_queue.cc:260] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.469579  1622 ts_tablet_manager.cc:1434] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 12:21:56.470607  1622 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:21:56.471717  1388 leader_election.cc:343] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.473376  1388 leader_election.cc:304] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.476822  1622 tablet_bootstrap.cc:654] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.479135  1853 raft_consensus.cc:2749] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.446460  1618 raft_consensus.cc:359] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.409695  1619 ts_tablet_manager.cc:1434] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.013s
I20260709 12:21:56.481976  1619 tablet_bootstrap.cc:492] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:56.482654  1618 raft_consensus.cc:385] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.483242  1618 raft_consensus.cc:740] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.484192  1618 consensus_queue.cc:260] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.451633  1866 raft_consensus.cc:2749] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.408746  1623 tablet_bootstrap.cc:654] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.489238  1633 raft_consensus.cc:493] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.489806  1633 raft_consensus.cc:515] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.490329  1846 consensus_queue.cc:1048] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.492746  1633 leader_election.cc:290] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:56.493530  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1ad92910454a39a4ebb1ea41c3c376" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:56.494212  1420 raft_consensus.cc:2468] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:56.494302  1618 ts_tablet_manager.cc:1434] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.050s	user 0.002s	sys 0.005s
I20260709 12:21:56.495316  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1ad92910454a39a4ebb1ea41c3c376" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:56.495358  1537 leader_election.cc:304] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:56.496132  1347 raft_consensus.cc:2468] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:56.496385  1633 raft_consensus.cc:2804] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.496781  1633 raft_consensus.cc:493] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.497153  1633 raft_consensus.cc:3060] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.499343  1618 tablet_bootstrap.cc:492] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.500691  1747 raft_consensus.cc:515] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.505167  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c312c9473d0e47e0816684df0c14a27f" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:56.506395  1622 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:56.507045  1622 ts_tablet_manager.cc:1403] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.000s
W20260709 12:21:56.510504  1386 leader_election.cc:343] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.514153  1622 raft_consensus.cc:359] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.515007  1622 raft_consensus.cc:385] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.516189  1622 raft_consensus.cc:740] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.519167  1947 consensus_queue.cc:1048] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.520815  1622 consensus_queue.cc:260] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.521965  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c312c9473d0e47e0816684df0c14a27f" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:56.522766  1497 raft_consensus.cc:3060] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.523610  1622 ts_tablet_manager.cc:1434] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:21:56.524811  1622 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:56.520841  1747 leader_election.cc:290] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:56.520009  1618 tablet_bootstrap.cc:654] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.529709  1846 raft_consensus.cc:493] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.532135  1846 raft_consensus.cc:515] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.503306  1619 tablet_bootstrap.cc:654] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.534474  1846 leader_election.cc:290] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:56.536566  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:56.537822  1570 raft_consensus.cc:2468] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:56.538215  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:56.538973  1423 raft_consensus.cc:2468] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:56.539116  1461 leader_election.cc:304] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:56.539971  1846 raft_consensus.cc:2804] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.540459  1846 raft_consensus.cc:493] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.540915  1846 raft_consensus.cc:3060] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.547410  1622 tablet_bootstrap.cc:654] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.545706  1623 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:56.565507  1623 ts_tablet_manager.cc:1403] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.167s	user 0.016s	sys 0.001s
I20260709 12:21:56.579917  1797 raft_consensus.cc:493] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.580813  1797 raft_consensus.cc:515] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.569658  1497 raft_consensus.cc:2468] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:56.584755  1618 tablet_bootstrap.cc:492] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.584522  1623 raft_consensus.cc:359] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.585386  1618 ts_tablet_manager.cc:1403] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.014s
I20260709 12:21:56.587072  1388 leader_election.cc:304] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.587369  1633 raft_consensus.cc:515] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.588581  1618 raft_consensus.cc:359] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.589591  1618 raft_consensus.cc:385] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.589509  1747 raft_consensus.cc:2804] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.590240  1747 raft_consensus.cc:697] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:56.591187  1747 consensus_queue.cc:237] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.585409  1623 raft_consensus.cc:385] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.592254  1623 raft_consensus.cc:740] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.592473  1619 tablet_bootstrap.cc:492] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:56.592993  1619 ts_tablet_manager.cc:1403] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.111s	user 0.012s	sys 0.000s
I20260709 12:21:56.596370  1622 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:56.603291  1622 ts_tablet_manager.cc:1403] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.018s
I20260709 12:21:56.606719  1633 leader_election.cc:290] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:56.609359  1622 raft_consensus.cc:359] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.598752  1618 raft_consensus.cc:740] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.604522  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1ad92910454a39a4ebb1ea41c3c376" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:56.610750  1348 raft_consensus.cc:3060] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.610913  1618 consensus_queue.cc:260] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.613322  1618 ts_tablet_manager.cc:1434] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:21:56.596019  1619 raft_consensus.cc:359] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.613881  1619 raft_consensus.cc:385] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.614203  1619 raft_consensus.cc:740] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.605216  1853 raft_consensus.cc:493] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.614843  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1ad92910454a39a4ebb1ea41c3c376" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:56.614964  1619 consensus_queue.cc:260] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.615577  1420 raft_consensus.cc:3060] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.615379  1853 raft_consensus.cc:515] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.617166  1618 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.618331  1853 leader_election.cc:290] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:56.620253  1797 leader_election.cc:290] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:56.623343  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:56.625849  1348 raft_consensus.cc:2468] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:56.634930  1618 tablet_bootstrap.cc:654] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.634776  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2236e9eab6544f1f9e89679486b7a6cf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.635744  1497 raft_consensus.cc:2468] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:56.635627  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:56.636854  1571 raft_consensus.cc:2468] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:56.637001  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2236e9eab6544f1f9e89679486b7a6cf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:56.593134  1623 consensus_queue.cc:260] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.640508  1623 ts_tablet_manager.cc:1434] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.000s
I20260709 12:21:56.641201  1535 leader_election.cc:304] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:21:56.641741  1623 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
W20260709 12:21:56.644294  1386 leader_election.cc:343] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.645447  1618 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.646101  1618 ts_tablet_manager.cc:1403] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.001s
I20260709 12:21:56.648093  1846 raft_consensus.cc:515] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:56.649112  1461 leader_election.cc:343] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.676062  1461 leader_election.cc:304] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.649437  1618 raft_consensus.cc:359] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.678402  1947 raft_consensus.cc:493] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.678745  1618 raft_consensus.cc:385] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.679080  1947 raft_consensus.cc:515] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.680094  1916 raft_consensus.cc:2804] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.680547  1916 raft_consensus.cc:493] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.680874  1916 raft_consensus.cc:3060] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.681986  1947 leader_election.cc:290] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:56.684235  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:56.686185  1535 leader_election.cc:343] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.689595  1916 raft_consensus.cc:515] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.690351  1774 raft_consensus.cc:493] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.690920  1774 raft_consensus.cc:515] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.693153  1774 leader_election.cc:290] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:56.694096  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:56.695294  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:56.695785  1418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:56.696116  1570 raft_consensus.cc:3060] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.679121  1618 raft_consensus.cc:740] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.695751  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984e201fabf540b6a2d1a83a6d186454" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.697839  1618 consensus_queue.cc:260] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.698907  1312 leader_election.cc:304] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.700253  1618 ts_tablet_manager.cc:1434] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.054s	user 0.010s	sys 0.000s
I20260709 12:21:56.700752  1774 raft_consensus.cc:2749] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.649765  1420 raft_consensus.cc:2468] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:56.652137  1670 raft_consensus.cc:2804] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.702034  1670 raft_consensus.cc:697] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:56.702625  1618 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.702932  1670 consensus_queue.cc:237] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.703792  1622 raft_consensus.cc:385] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.669704  1623 tablet_bootstrap.cc:654] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.664151  1973 raft_consensus.cc:493] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.712538  1622 raft_consensus.cc:740] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.710253  1619 ts_tablet_manager.cc:1434] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.117s	user 0.013s	sys 0.021s
I20260709 12:21:56.716889  1622 consensus_queue.cc:260] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.715133  1973 raft_consensus.cc:515] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.719939  1622 ts_tablet_manager.cc:1434] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.113s	user 0.004s	sys 0.002s
I20260709 12:21:56.670113  1388 leader_election.cc:304] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
I20260709 12:21:56.717403  1619 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:56.720868  1973 leader_election.cc:290] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
W20260709 12:21:56.710769  1461 leader_election.cc:343] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.726799  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bc80953a524e6f844c37036c559c68" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:56.727468  1622 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:56.728467  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:56.729137  1571 raft_consensus.cc:3060] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:56.733265  1388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.67:43099, user_credentials={real_user=slave}} blocked reactor thread for 63257.5us
I20260709 12:21:56.716180  1618 tablet_bootstrap.cc:654] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.729039  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bc80953a524e6f844c37036c559c68" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.737869  1980 raft_consensus.cc:2804] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.738448  1980 raft_consensus.cc:493] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.738878  1980 raft_consensus.cc:3060] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.717801  1916 leader_election.cc:290] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:56.745306  1570 raft_consensus.cc:2468] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:56.746587  1461 leader_election.cc:304] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.747665  1916 raft_consensus.cc:2804] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.748248  1916 raft_consensus.cc:697] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:56.748623  1980 raft_consensus.cc:515] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.750052  1619 tablet_bootstrap.cc:654] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:56.738771  1535 leader_election.cc:343] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.751363  1916 consensus_queue.cc:237] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.754031  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2236e9eab6544f1f9e89679486b7a6cf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:56.705204  1846 leader_election.cc:290] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:56.755123  1980 leader_election.cc:290] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
W20260709 12:21:56.757503  1386 leader_election.cc:343] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.758579  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:56.759251  1422 raft_consensus.cc:2468] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:56.755502  1853 raft_consensus.cc:493] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.764832  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:56.765911  1420 raft_consensus.cc:3060] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.765803  1853 raft_consensus.cc:515] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:56.769555  1537 leader_election.cc:343] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.770056  1537 leader_election.cc:304] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.770978  1670 raft_consensus.cc:2749] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.772428  1853 leader_election.cc:290] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:56.742151  1571 raft_consensus.cc:2468] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:56.764429  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2236e9eab6544f1f9e89679486b7a6cf" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:56.776441  1497 raft_consensus.cc:3060] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.775923  1244 catalog_manager.cc:5697] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.778221  1420 raft_consensus.cc:2468] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:56.774936  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
W20260709 12:21:56.754776  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:56.737833  1622 tablet_bootstrap.cc:654] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:56.779836  1386 leader_election.cc:343] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.780501  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.781920  1846 raft_consensus.cc:493] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.782305  1623 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:56.782832  1623 ts_tablet_manager.cc:1403] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.141s	user 0.014s	sys 0.004s
I20260709 12:21:56.758004  1240 catalog_manager.cc:5697] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.785517  1463 leader_election.cc:304] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:56.789335  1796 raft_consensus.cc:2804] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.791549  1796 raft_consensus.cc:697] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:56.792507  1796 consensus_queue.cc:237] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.792972  1497 raft_consensus.cc:2468] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:56.785799  1623 raft_consensus.cc:359] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.795012  1623 raft_consensus.cc:385] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.795440  1623 raft_consensus.cc:740] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.782441  1846 raft_consensus.cc:515] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.796131  1623 consensus_queue.cc:260] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.797663  1846 leader_election.cc:290] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
W20260709 12:21:56.788071  1388 leader_election.cc:343] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.799178  1388 leader_election.cc:304] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.799628  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:56.800117  1987 raft_consensus.cc:2749] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.800726  1388 leader_election.cc:304] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee
W20260709 12:21:56.801573  1461 leader_election.cc:343] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.801728  1987 raft_consensus.cc:2804] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.788213  1618 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.802361  1987 raft_consensus.cc:697] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:56.802821  1618 ts_tablet_manager.cc:1403] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.101s	user 0.007s	sys 0.006s
I20260709 12:21:56.802717  1537 leader_election.cc:304] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.803473  1987 consensus_queue.cc:237] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.805457  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:56.806115  1420 raft_consensus.cc:2468] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:56.807262  1463 leader_election.cc:304] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.807621  1618 raft_consensus.cc:359] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.808285  1619 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:56.808440  1618 raft_consensus.cc:385] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.808781  1619 ts_tablet_manager.cc:1403] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.000s
I20260709 12:21:56.808634  1796 raft_consensus.cc:2804] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.809728  1796 raft_consensus.cc:493] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.810268  1796 raft_consensus.cc:3060] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.810745  1670 raft_consensus.cc:2804] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.811199  1670 raft_consensus.cc:493] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.811550  1670 raft_consensus.cc:3060] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.814703  1622 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:56.815310  1622 ts_tablet_manager.cc:1403] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.088s	user 0.008s	sys 0.020s
I20260709 12:21:56.819849  1796 raft_consensus.cc:515] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.824549  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:56.808735  1618 raft_consensus.cc:740] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.825788  1622 raft_consensus.cc:359] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.826120  1618 consensus_queue.cc:260] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.826808  1622 raft_consensus.cc:385] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.827126  1622 raft_consensus.cc:740] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.828173  1347 raft_consensus.cc:3060] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.827945  1670 raft_consensus.cc:515] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.828557  1796 leader_election.cc:290] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:56.829828  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:56.830104  1670 leader_election.cc:290] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:56.830475  1420 raft_consensus.cc:3060] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:56.832412  1461 leader_election.cc:343] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.833338  1618 ts_tablet_manager.cc:1434] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:21:56.798354  1623 ts_tablet_manager.cc:1434] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.000s
I20260709 12:21:56.834632  1618 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.827965  1622 consensus_queue.cc:260] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.835582  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
W20260709 12:21:56.837256  1535 leader_election.cc:343] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:56.841158  1619 raft_consensus.cc:359] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.842129  1619 raft_consensus.cc:385] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.842129  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:56.842887  1422 raft_consensus.cc:3060] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.843459  1622 ts_tablet_manager.cc:1434] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:21:56.844558  1622 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:56.846614  1619 raft_consensus.cc:740] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.847414  1619 consensus_queue.cc:260] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.849589  1619 ts_tablet_manager.cc:1434] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 12:21:56.835619  1623 tablet_bootstrap.cc:492] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:56.850587  1619 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:56.860015  1422 raft_consensus.cc:2468] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:56.861588  1537 leader_election.cc:304] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.854115  1420 raft_consensus.cc:2468] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:56.860387  1618 tablet_bootstrap.cc:654] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.863667  1463 leader_election.cc:304] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:56.865000  1916 raft_consensus.cc:2804] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.866087  1916 raft_consensus.cc:697] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:56.871614  1241 catalog_manager.cc:5697] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.874735  1670 raft_consensus.cc:2804] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.874066  1916 consensus_queue.cc:237] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:56.875680  1670 raft_consensus.cc:697] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:56.876811  1670 consensus_queue.cc:237] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:56.880707  1618 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.884018  1618 ts_tablet_manager.cc:1403] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.000s
I20260709 12:21:56.861663  1622 tablet_bootstrap.cc:654] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.887982  1623 tablet_bootstrap.cc:654] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.888207  1618 raft_consensus.cc:359] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.889003  1618 raft_consensus.cc:385] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.889353  1618 raft_consensus.cc:740] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.890338  1618 consensus_queue.cc:260] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.893088  1618 ts_tablet_manager.cc:1434] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:21:56.895774  1618 tablet_bootstrap.cc:492] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.893227  1774 raft_consensus.cc:493] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.897573  1774 raft_consensus.cc:515] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.907418  1618 tablet_bootstrap.cc:654] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.914384  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.919852  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086651b7e7954168abc2acd51a3ed43d" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:56.926541  1866 raft_consensus.cc:493] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.927446  1623 tablet_bootstrap.cc:492] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:56.927254  1866 raft_consensus.cc:515] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:56.928088  1623 ts_tablet_manager.cc:1403] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.004s
I20260709 12:21:56.928633  1618 tablet_bootstrap.cc:492] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:56.929351  1618 ts_tablet_manager.cc:1403] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.004s
I20260709 12:21:56.930727  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:56.931422  1310 leader_election.cc:304] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.931579  1623 raft_consensus.cc:359] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.933086  1913 raft_consensus.cc:2749] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.933261  1623 raft_consensus.cc:385] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.933605  1623 raft_consensus.cc:740] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.934512  1623 consensus_queue.cc:260] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.937170  1623 ts_tablet_manager.cc:1434] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:21:56.938294  1623 tablet_bootstrap.cc:492] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:56.940389  1618 raft_consensus.cc:359] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.941998  1618 raft_consensus.cc:385] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.942346  1618 raft_consensus.cc:740] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.943378  1618 consensus_queue.cc:260] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.945596  1623 tablet_bootstrap.cc:654] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.946105  1618 ts_tablet_manager.cc:1434] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:21:56.947520  1618 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:56.948138  1774 leader_election.cc:290] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:56.950088  1866 leader_election.cc:290] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:56.952862  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "629287ce1cfc44a0b85fcc0dcbde5c07" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:56.953987  1570 raft_consensus.cc:2393] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate e8246e0558124e98a58932ad7f4676e3 in this term.
I20260709 12:21:56.955863  1310 leader_election.cc:304] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:56.958410  1618 tablet_bootstrap.cc:654] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.961190  1623 tablet_bootstrap.cc:492] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:56.961905  1623 ts_tablet_manager.cc:1403] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.005s
I20260709 12:21:56.966274  1623 raft_consensus.cc:359] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.967056  1623 raft_consensus.cc:385] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.967362  1623 raft_consensus.cc:740] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.968242  1623 consensus_queue.cc:260] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:56.970182  1866 raft_consensus.cc:3060] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.975041  1623 ts_tablet_manager.cc:1434] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:21:56.976387  1623 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:56.977918  1243 catalog_manager.cc:5697] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.998001  1348 raft_consensus.cc:3060] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.989065  1623 tablet_bootstrap.cc:654] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.992852  1618 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:57.011606  1618 ts_tablet_manager.cc:1403] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.007s
I20260709 12:21:57.014969  1618 raft_consensus.cc:359] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:57.015753  1618 raft_consensus.cc:385] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.016033  1618 raft_consensus.cc:740] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.017023  1618 consensus_queue.cc:260] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:57.020175  1618 ts_tablet_manager.cc:1434] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:21:57.021378  1618 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:57.038879  1796 raft_consensus.cc:493] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.039690  1796 raft_consensus.cc:515] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.042378  1796 leader_election.cc:290] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:57.053694  1619 tablet_bootstrap.cc:654] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:57.059728  1947 raft_consensus.cc:493] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.060652  1947 raft_consensus.cc:515] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:57.063313  1618 tablet_bootstrap.cc:654] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:57.063632  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bc80953a524e6f844c37036c559c68" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:57.066746  1461 leader_election.cc:343] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.048291  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bc80953a524e6f844c37036c559c68" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:57.071794  1570 raft_consensus.cc:2468] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:57.072001  1987 consensus_queue.cc:1048] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:57.073714  1774 raft_consensus.cc:493] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.074344  1774 raft_consensus.cc:515] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:57.076777  1774 leader_election.cc:290] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:57.081338  1461 leader_election.cc:304] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:57.080838  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c312c9473d0e47e0816684df0c14a27f" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:57.077903  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c312c9473d0e47e0816684df0c14a27f" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:57.077637  1973 raft_consensus.cc:493] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.087798  1346 raft_consensus.cc:3060] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.088339  1846 raft_consensus.cc:2804] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:57.097162  1846 raft_consensus.cc:493] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.097568  1846 raft_consensus.cc:3060] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.102299  1622 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:57.102857  1622 ts_tablet_manager.cc:1403] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.259s	user 0.019s	sys 0.018s
I20260709 12:21:57.105566  1622 raft_consensus.cc:359] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.106215  1622 raft_consensus.cc:385] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.107792  1312 leader_election.cc:304] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:57.110419  1866 raft_consensus.cc:2749] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:57.093134  1973 raft_consensus.cc:515] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.113919  1973 leader_election.cc:290] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:57.114141  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa446bfc89d4e82bd94dd6ce5699591" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:57.115542  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa446bfc89d4e82bd94dd6ce5699591" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
W20260709 12:21:57.115828  1535 leader_election.cc:343] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.116464  1420 raft_consensus.cc:2468] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:57.117926  1623 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:57.119457  1623 ts_tablet_manager.cc:1403] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.143s	user 0.021s	sys 0.010s
I20260709 12:21:57.120698  1622 raft_consensus.cc:740] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.123883  1537 leader_election.cc:304] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:57.123719  1241 catalog_manager.cc:5697] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:57.121587  1622 consensus_queue.cc:260] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.128352  1241 catalog_manager.cc:5697] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:57.134060  1240 catalog_manager.cc:5697] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:57.149792  1622 ts_tablet_manager.cc:1434] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 12:21:57.139124  1973 raft_consensus.cc:2804] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:57.154618  1973 raft_consensus.cc:493] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.155007  1973 raft_consensus.cc:3060] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.160142  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:57.160833  1498 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:57.161047  1846 raft_consensus.cc:515] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.162767  1537 leader_election.cc:304] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:57.154799  1947 leader_election.cc:290] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:57.164786  1633 raft_consensus.cc:2804] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:57.172204  1633 raft_consensus.cc:493] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.172648  1633 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.173655  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bc80953a524e6f844c37036c559c68" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:57.174348  1345 raft_consensus.cc:3060] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.174643  1913 raft_consensus.cc:2749] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:57.166450  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:57.177661  1422 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:57.168996  1846 leader_election.cc:290] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:57.170382  1623 raft_consensus.cc:359] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.171397  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bc80953a524e6f844c37036c559c68" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:57.184144  1570 raft_consensus.cc:3060] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.171182  1982 consensus_queue.cc:1048] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:57.186944  1947 raft_consensus.cc:493] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.187602  1947 raft_consensus.cc:515] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:57.189961  1947 leader_election.cc:290] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:57.192211  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2236e9eab6544f1f9e89679486b7a6cf" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:57.194460  1496 raft_consensus.cc:2393] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76217635dae3410b96baf380d386cbee in current term 1: Already voted for candidate 95782952bdee421094cada4ea955015d in this term.
I20260709 12:21:57.183586  1623 raft_consensus.cc:385] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.195523  1623 raft_consensus.cc:740] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.195847  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2236e9eab6544f1f9e89679486b7a6cf" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:57.196520  1623 consensus_queue.cc:260] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.206913  1345 raft_consensus.cc:2468] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:57.208926  1537 leader_election.cc:304] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:57.209149  1461 leader_election.cc:304] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:57.209610  1623 ts_tablet_manager.cc:1434] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.000s
I20260709 12:21:57.210621  1947 raft_consensus.cc:3060] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.210696  1916 raft_consensus.cc:2804] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:57.212438  1916 raft_consensus.cc:697] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:57.213830  1623 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:57.213546  1916 consensus_queue.cc:237] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.219743  1622 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:57.219846  1846 raft_consensus.cc:493] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.228590  1846 raft_consensus.cc:515] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.229604  1618 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:57.230211  1618 ts_tablet_manager.cc:1403] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.209s	user 0.031s	sys 0.019s
I20260709 12:21:57.231015  1846 leader_election.cc:290] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:57.233495  1618 raft_consensus.cc:359] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.235747  1618 raft_consensus.cc:385] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.236033  1618 raft_consensus.cc:740] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.236891  1618 consensus_queue.cc:260] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.241933  1618 ts_tablet_manager.cc:1434] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.001s
I20260709 12:21:57.242823  1846 raft_consensus.cc:493] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.251415  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:57.255807  1618 tablet_bootstrap.cc:492] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:57.257867  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:57.258837  1847 raft_consensus.cc:493] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.259414  1847 raft_consensus.cc:515] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.261737  1847 leader_election.cc:290] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
W20260709 12:21:57.262387  1461 leader_election.cc:343] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.251403  1846 raft_consensus.cc:515] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:21:57.271193  1461 leader_election.cc:343] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.273247  1461 leader_election.cc:304] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:57.276381  1847 raft_consensus.cc:2749] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:57.277755  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:57.286175  1846 leader_election.cc:290] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:57.288781  1344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:57.289855  1343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:57.291217  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
W20260709 12:21:57.295382  1463 leader_election.cc:343] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:57.297957  1535 consensus_peers.cc:597] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:57.303622  1570 raft_consensus.cc:2468] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:57.304939  1623 tablet_bootstrap.cc:654] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:57.313586  1461 leader_election.cc:343] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.314224  1461 leader_election.cc:304] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
W20260709 12:21:57.315625  1461 leader_election.cc:343] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:57.316402  1461 leader_election.cc:343] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.316872  1461 leader_election.cc:304] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:57.323355  1846 raft_consensus.cc:2749] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:57.322428  1796 raft_consensus.cc:2749] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:57.328608  1982 consensus_queue.cc:1048] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:57.365303  1947 raft_consensus.cc:2749] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:57.365762  1633 raft_consensus.cc:515] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:57.367846  1619 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:57.368436  1633 leader_election.cc:290] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:57.368335  1973 raft_consensus.cc:515] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.371951  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa446bfc89d4e82bd94dd6ce5699591" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
W20260709 12:21:57.373821  1535 leader_election.cc:343] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.376267  1618 tablet_bootstrap.cc:654] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:57.368448  1619 ts_tablet_manager.cc:1403] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.518s	user 0.056s	sys 0.043s
I20260709 12:21:57.380095  1619 raft_consensus.cc:359] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:57.381227  1622 tablet_bootstrap.cc:654] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:57.383903  1619 raft_consensus.cc:385] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.388010  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:57.387830  2013 consensus_queue.cc:1048] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:57.388897  1423 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.384333  1619 raft_consensus.cc:740] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.393810  2013 consensus_queue.cc:1048] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:57.394363  1619 consensus_queue.cc:260] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:57.397017  1619 ts_tablet_manager.cc:1434] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 12:21:57.398151  1619 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
W20260709 12:21:57.405296  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 114132us
I20260709 12:21:57.407538  1973 leader_election.cc:290] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:57.420042  1423 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:57.416527  1980 consensus_queue.cc:1048] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:21:57.432926  1498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:57.431864  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa446bfc89d4e82bd94dd6ce5699591" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:57.436283  1498 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.436589  1423 raft_consensus.cc:3060] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:57.437103  1537 leader_election.cc:304] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:57.442507  1670 raft_consensus.cc:2804] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:57.454032  1423 raft_consensus.cc:2468] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:57.427255  2014 consensus_queue.cc:1048] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:21:57.469441  1498 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:57.476032  1670 raft_consensus.cc:697] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:57.477316  1670 consensus_queue.cc:237] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:57.491693  1537 leader_election.cc:304] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:57.502470  1623 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:57.503676  1623 ts_tablet_manager.cc:1403] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.290s	user 0.019s	sys 0.008s
W20260709 12:21:57.553640  1461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 92052.7us
W20260709 12:21:57.557475  1537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 65918us
W20260709 12:21:57.513048  2017 log.cc:927] Time spent T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:21:57.568795  2020 log.cc:927] Time spent T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:21:57.563805  1846 consensus_queue.cc:1048] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 12:21:57.566622  1973 raft_consensus.cc:493] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.589622  1623 raft_consensus.cc:359] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.600582  1623 raft_consensus.cc:385] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:57.568789  2019 log.cc:927] Time spent T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:21:57.600993  1623 raft_consensus.cc:740] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.607013  1623 consensus_queue.cc:260] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:57.614802  1623 ts_tablet_manager.cc:1434] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.111s	user 0.008s	sys 0.002s
I20260709 12:21:57.608131  1240 catalog_manager.cc:5697] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:57.627208  2014 consensus_queue.cc:1048] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:57.628180  1619 tablet_bootstrap.cc:654] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:57.602975  2025 raft_consensus.cc:2804] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:57.616730  1973 raft_consensus.cc:515] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.645237  1623 tablet_bootstrap.cc:492] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:57.646080  2025 raft_consensus.cc:697] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:57.705080  1973 leader_election.cc:290] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:57.708155  2025 consensus_queue.cc:237] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.734606  2023 consensus_queue.cc:1048] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:57.735739  1618 tablet_bootstrap.cc:492] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:57.736529  1618 ts_tablet_manager.cc:1403] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.481s	user 0.020s	sys 0.040s
I20260709 12:21:57.738390  1622 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:57.740003  1622 ts_tablet_manager.cc:1403] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.521s	user 0.023s	sys 0.011s
I20260709 12:21:57.751287  1623 tablet_bootstrap.cc:654] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:57.753019  1618 raft_consensus.cc:359] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:57.755312  1618 raft_consensus.cc:385] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.755780  1618 raft_consensus.cc:740] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
W20260709 12:21:57.758633  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 313876us
I20260709 12:21:57.747399  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:57.768941  1619 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:57.769642  1619 ts_tablet_manager.cc:1403] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.372s	user 0.020s	sys 0.010s
I20260709 12:21:57.771299  1618 consensus_queue.cc:260] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:57.773298  1619 raft_consensus.cc:359] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.774262  1619 raft_consensus.cc:385] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.748610  1947 consensus_queue.cc:1048] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:21:57.773247  1618 ts_tablet_manager.cc:1434] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.005s
I20260709 12:21:57.777194  1619 raft_consensus.cc:740] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.778258  1618 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
I20260709 12:21:57.778210  1619 consensus_queue.cc:260] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:57.781379  1619 ts_tablet_manager.cc:1434] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:21:57.801328  2022 raft_consensus.cc:493] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:57.801869  1537 leader_election.cc:343] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.805298  1619 tablet_bootstrap.cc:492] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:57.802028  2022 raft_consensus.cc:515] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:21:57.799484  1461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 76334us
I20260709 12:21:57.832139  2016 consensus_queue.cc:1048] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:21:57.836089  2022 leader_election.cc:290] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:57.848577  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:57.844866  1622 raft_consensus.cc:359] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:57.853150  1622 raft_consensus.cc:385] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:57.856321  1618 tablet_bootstrap.cc:654] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:57.853636  1622 raft_consensus.cc:740] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:57.857841  1622 consensus_queue.cc:260] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:57.869206  1619 tablet_bootstrap.cc:654] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:57.870357  1463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 43938.6us
I20260709 12:21:57.884675  1622 ts_tablet_manager.cc:1434] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.144s	user 0.008s	sys 0.002s
W20260709 12:21:57.906567  1312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 92990.2us
I20260709 12:21:57.960711  1622 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:57.914117  2025 raft_consensus.cc:493] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:57.976950  1386 leader_election.cc:343] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:57.985085  2037 raft_consensus.cc:493] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.970423  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:21:58.008945  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:57.985715  2037 raft_consensus.cc:515] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.037768  1797 raft_consensus.cc:493] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:58.081946  1461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:37337, user_credentials={real_user=slave}} blocked reactor thread for 96309.6us
W20260709 12:21:58.088668  1461 consensus_peers.cc:597] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:57.984364  2025 raft_consensus.cc:515] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.101958  1622 tablet_bootstrap.cc:654] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.087069  1797 raft_consensus.cc:515] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:58.146052  1623 tablet_bootstrap.cc:492] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:58.146793  1623 ts_tablet_manager.cc:1403] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.514s	user 0.014s	sys 0.005s
W20260709 12:21:58.150071  2044 log.cc:927] Time spent T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
W20260709 12:21:58.159448  2048 log.cc:927] Time spent T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:21:58.159871  2050 consensus_queue.cc:1048] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
I20260709 12:21:58.162170  1344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:58.164456  1618 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:58.165026  1618 ts_tablet_manager.cc:1403] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.387s	user 0.013s	sys 0.008s
I20260709 12:21:58.168234  1618 raft_consensus.cc:359] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.168844  2037 leader_election.cc:290] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:58.168895  1618 raft_consensus.cc:385] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.169965  1618 raft_consensus.cc:740] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.177063  2025 leader_election.cc:290] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:58.181136  1618 consensus_queue.cc:260] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.153182  1866 raft_consensus.cc:493] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.179757  1797 leader_election.cc:290] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:58.203765  1866 raft_consensus.cc:515] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.213057  1866 leader_election.cc:290] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:21:58.214102  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:58.215646  1623 raft_consensus.cc:359] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:58.216391  1623 raft_consensus.cc:385] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.216660  1623 raft_consensus.cc:740] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.217367  1623 consensus_queue.cc:260] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
W20260709 12:21:58.215662  1388 leader_election.cc:343] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.218679  1388 leader_election.cc:304] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:58.220146  2024 raft_consensus.cc:2749] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:58.226054  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89da9bea35044babd7f5631e0e7afd8" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:58.230576  1623 ts_tablet_manager.cc:1434] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.079s	user 0.002s	sys 0.005s
I20260709 12:21:58.231786  1623 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:58.233631  1312 leader_election.cc:304] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:58.237452  1618 ts_tablet_manager.cc:1434] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.000s
I20260709 12:21:58.247844  1339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:58.254341  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4742a96787194d49a2f191cdd0ef8642" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:58.255336  1566 raft_consensus.cc:2468] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:58.255679  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:58.256510  1493 raft_consensus.cc:2468] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:58.260244  2053 raft_consensus.cc:2749] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:58.263132  1243 catalog_manager.cc:5697] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:58.278059  1386 leader_election.cc:343] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.273785  1618 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3: Bootstrap starting.
W20260709 12:21:58.276979  1535 leader_election.cc:343] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.281571  1535 leader_election.cc:304] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:21:58.286418  1619 tablet_bootstrap.cc:492] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:58.286995  1619 ts_tablet_manager.cc:1403] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.482s	user 0.019s	sys 0.006s
I20260709 12:21:58.283528  2025 raft_consensus.cc:2749] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:58.292224  1619 raft_consensus.cc:359] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:58.292932  1619 raft_consensus.cc:385] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.293182  1619 raft_consensus.cc:740] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.293898  1619 consensus_queue.cc:260] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:58.294551  1340 raft_consensus.cc:3060] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.300843  1619 ts_tablet_manager.cc:1434] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:21:58.302040  1619 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:58.308434  1618 tablet_bootstrap.cc:654] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.327569  1633 consensus_queue.cc:1048] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:58.327939  1537 leader_election.cc:304] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:58.336265  1633 raft_consensus.cc:2804] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.336716  1633 raft_consensus.cc:493] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.337013  1633 raft_consensus.cc:3060] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.336265  1461 leader_election.cc:304] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:58.345713  1619 tablet_bootstrap.cc:654] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.343055  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:58.347807  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4742a96787194d49a2f191cdd0ef8642" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:58.348821  1423 raft_consensus.cc:2468] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:58.341653  2023 raft_consensus.cc:2804] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.380527  1633 raft_consensus.cc:515] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.372449  2023 raft_consensus.cc:493] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.405203  1623 tablet_bootstrap.cc:654] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.406613  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:58.411315  1633 leader_election.cc:290] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:58.396586  2023 raft_consensus.cc:3060] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.415197  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:58.423817  1618 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3: No bootstrap required, opened a new log
I20260709 12:21:58.435906  2064 consensus_queue.cc:1048] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:58.426764  1386 leader_election.cc:343] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76217635dae3410b96baf380d386cbee (127.23.217.68:37405): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.445693  1618 ts_tablet_manager.cc:1403] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3: Time spent bootstrapping tablet: real 0.205s	user 0.010s	sys 0.014s
I20260709 12:21:58.445755  1386 leader_election.cc:304] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:21:58.455099  1853 raft_consensus.cc:2749] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:58.454408  1495 raft_consensus.cc:3060] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.481602  1622 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:58.482213  1622 ts_tablet_manager.cc:1403] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.560s	user 0.043s	sys 0.025s
I20260709 12:21:58.483734  1618 raft_consensus.cc:359] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.484822  1618 raft_consensus.cc:385] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.485188  1618 raft_consensus.cc:740] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Initialized, Role: FOLLOWER
W20260709 12:21:58.486591  1535 leader_election.cc:343] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.484892  1622 raft_consensus.cc:359] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.487385  1622 raft_consensus.cc:385] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.487715  1622 raft_consensus.cc:740] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.486150  1495 raft_consensus.cc:2468] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:58.486176  1618 consensus_queue.cc:260] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.507264  1622 consensus_queue.cc:260] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.523301  1622 ts_tablet_manager.cc:1434] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.002s
I20260709 12:21:58.528615  1341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:58.528990  1618 ts_tablet_manager.cc:1434] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.003s
I20260709 12:21:58.544929  1619 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:58.545528  1619 ts_tablet_manager.cc:1403] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.244s	user 0.014s	sys 0.022s
W20260709 12:21:58.545938  1535 consensus_peers.cc:597] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.554244  1537 leader_election.cc:304] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:58.555605  1633 raft_consensus.cc:2804] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.556046  1633 raft_consensus.cc:697] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:58.556768  1633 consensus_queue.cc:237] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
W20260709 12:21:58.557763  1535 leader_election.cc:343] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.567866  2025 consensus_queue.cc:1048] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:58.570680  1619 raft_consensus.cc:359] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.592810  1619 raft_consensus.cc:385] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.593178  1619 raft_consensus.cc:740] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.593930  1619 consensus_queue.cc:260] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.571732  2023 raft_consensus.cc:515] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:58.578596  1623 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:58.598174  1623 ts_tablet_manager.cc:1403] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.367s	user 0.010s	sys 0.026s
I20260709 12:21:58.599604  2023 leader_election.cc:290] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:58.579577  1622 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:58.580013  2025 consensus_queue.cc:1048] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:58.583673  1797 raft_consensus.cc:493] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.589684  1345 raft_consensus.cc:3060] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.609498  1619 ts_tablet_manager.cc:1434] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.000s
I20260709 12:21:58.617146  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4742a96787194d49a2f191cdd0ef8642" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:58.618221  1417 raft_consensus.cc:3060] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.626174  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4742a96787194d49a2f191cdd0ef8642" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:58.627199  1568 raft_consensus.cc:3060] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.618741  1623 raft_consensus.cc:359] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.631386  1623 raft_consensus.cc:385] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.631824  1623 raft_consensus.cc:740] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.632901  1623 consensus_queue.cc:260] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.615145  1797 raft_consensus.cc:515] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.645021  1342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:58.646924  1461 leader_election.cc:343] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.647393  1623 ts_tablet_manager.cc:1434] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.049s	user 0.011s	sys 0.000s
I20260709 12:21:58.646898  1797 leader_election.cc:290] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:58.659650  1619 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d: Bootstrap starting.
I20260709 12:21:58.659859  1623 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:58.672315  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:58.681560  1424 raft_consensus.cc:2468] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:58.688941  1619 tablet_bootstrap.cc:654] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.708897  1622 tablet_bootstrap.cc:654] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.708716  1463 leader_election.cc:304] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:58.689023  1568 raft_consensus.cc:2468] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:58.720017  1461 leader_election.cc:304] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:58.709964  2033 raft_consensus.cc:493] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.726819  2057 raft_consensus.cc:2804] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.728701  2033 raft_consensus.cc:515] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.731041  2033 leader_election.cc:290] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:58.733323  1344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cda1629f4344ad8b1ebed68a83d0418" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:58.735565  1535 leader_election.cc:343] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.735975  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cda1629f4344ad8b1ebed68a83d0418" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:58.736641  1495 raft_consensus.cc:2468] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:58.738791  1537 leader_election.cc:304] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:58.738936  2057 raft_consensus.cc:697] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:58.740059  2025 raft_consensus.cc:2804] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.740499  2025 raft_consensus.cc:493] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.740278  2057 consensus_queue.cc:237] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:21:58.744877  1417 raft_consensus.cc:2468] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:58.721216  1846 raft_consensus.cc:2804] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.740906  2025 raft_consensus.cc:3060] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.747232  1623 tablet_bootstrap.cc:654] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.771238  1846 raft_consensus.cc:493] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.794338  1243 catalog_manager.cc:5697] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.801574  1243 catalog_manager.cc:5697] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.788167  1846 raft_consensus.cc:3060] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.817993  1619 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d: No bootstrap required, opened a new log
I20260709 12:21:58.818573  1619 ts_tablet_manager.cc:1403] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d: Time spent bootstrapping tablet: real 0.165s	user 0.023s	sys 0.026s
I20260709 12:21:58.838948  1619 raft_consensus.cc:359] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.839695  1619 raft_consensus.cc:385] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.839988  1619 raft_consensus.cc:740] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.843458  2025 raft_consensus.cc:515] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.845590  2025 leader_election.cc:290] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:58.848842  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cda1629f4344ad8b1ebed68a83d0418" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:58.849066  1846 raft_consensus.cc:515] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.849535  1494 raft_consensus.cc:3060] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.851910  1619 consensus_queue.cc:260] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.848728  1340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cda1629f4344ad8b1ebed68a83d0418" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:58.855068  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:58.855718  1419 raft_consensus.cc:3060] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.855931  1623 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:58.862835  1622 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:58.863366  1622 ts_tablet_manager.cc:1403] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.284s	user 0.027s	sys 0.005s
I20260709 12:21:58.863739  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
W20260709 12:21:58.865306  1461 leader_election.cc:343] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.865892  1622 raft_consensus.cc:359] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.870592  1622 raft_consensus.cc:385] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.853585  1619 ts_tablet_manager.cc:1434] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 12:21:58.867669  1846 leader_election.cc:290] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
W20260709 12:21:58.868314  1535 leader_election.cc:343] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.878213  1494 raft_consensus.cc:2468] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:58.856426  1623 ts_tablet_manager.cc:1403] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.203s	user 0.022s	sys 0.002s
I20260709 12:21:58.881301  1623 raft_consensus.cc:359] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.881834  1623 raft_consensus.cc:385] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.882074  1623 raft_consensus.cc:740] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.873755  1419 raft_consensus.cc:2468] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:58.870893  1622 raft_consensus.cc:740] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.882745  1623 consensus_queue.cc:260] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.883749  1463 leader_election.cc:304] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:58.885097  1846 raft_consensus.cc:2804] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.884764  1622 consensus_queue.cc:260] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.885780  1846 raft_consensus.cc:697] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:58.886853  1846 consensus_queue.cc:237] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.887853  1622 ts_tablet_manager.cc:1434] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.003s
I20260709 12:21:58.887395  1244 catalog_manager.cc:5697] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.889104  1622 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:58.885118  1623 ts_tablet_manager.cc:1434] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:21:58.893201  1623 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee: Bootstrap starting.
I20260709 12:21:58.895975  1537 leader_election.cc:304] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:58.897245  2025 raft_consensus.cc:2804] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.897782  2025 raft_consensus.cc:697] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:58.898681  2025 consensus_queue.cc:237] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.907552  1622 tablet_bootstrap.cc:654] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.908931  1623 tablet_bootstrap.cc:654] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.923820  1243 catalog_manager.cc:5697] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:58.926944  1622 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:58.927613  1622 ts_tablet_manager.cc:1403] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.000s
I20260709 12:21:58.929598  2033 raft_consensus.cc:493] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.930186  2033 raft_consensus.cc:515] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.931519  1623 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee: No bootstrap required, opened a new log
I20260709 12:21:58.932111  1623 ts_tablet_manager.cc:1403] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.011s
I20260709 12:21:58.932605  2033 leader_election.cc:290] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:58.933108  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:58.933784  1494 raft_consensus.cc:2468] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
I20260709 12:21:58.929596  1244 catalog_manager.cc:5697] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.934177  1622 raft_consensus.cc:359] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.934964  1622 raft_consensus.cc:385] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.935242  1622 raft_consensus.cc:740] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.935643  1623 raft_consensus.cc:359] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.935995  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:58.936388  1623 raft_consensus.cc:385] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.936180  1622 consensus_queue.cc:260] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.936744  1623 raft_consensus.cc:740] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.937567  1623 consensus_queue.cc:260] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.938795  1622 ts_tablet_manager.cc:1434] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:21:58.939980  1623 ts_tablet_manager.cc:1434] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:58.940035  1537 leader_election.cc:304] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:58.941068  2033 raft_consensus.cc:2804] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.941452  2033 raft_consensus.cc:493] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:58.941717  1535 leader_election.cc:343] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.941828  2033 raft_consensus.cc:3060] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.950953  1622 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:58.958079  1622 tablet_bootstrap.cc:654] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.970441  1622 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:58.971073  1622 ts_tablet_manager.cc:1403] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 12:21:58.974538  1622 raft_consensus.cc:359] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.975229  1622 raft_consensus.cc:385] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.967144  2033 raft_consensus.cc:515] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:58.975508  1622 raft_consensus.cc:740] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.976460  1622 consensus_queue.cc:260] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.977871  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:21:58.978560  1494 raft_consensus.cc:3060] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.979283  1622 ts_tablet_manager.cc:1434] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:21:58.982184  1622 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:58.985587  1494 raft_consensus.cc:2468] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:58.987107  1537 leader_election.cc:304] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:58.988032  2025 raft_consensus.cc:2804] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.988509  2025 raft_consensus.cc:697] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:58.989205  1622 tablet_bootstrap.cc:654] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.989141  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:58.989378  2025 consensus_queue.cc:237] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
W20260709 12:21:58.991611  1535 leader_election.cc:343] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.997979  2074 raft_consensus.cc:493] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.998245  2057 raft_consensus.cc:493] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.998780  2074 raft_consensus.cc:515] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:58.998888  2057 raft_consensus.cc:515] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:58.997201  2033 leader_election.cc:290] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:59.003893  2074 leader_election.cc:290] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:59.004251  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:59.005009  1568 raft_consensus.cc:2468] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:59.005784  2057 leader_election.cc:290] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:59.007113  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:59.007407  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:59.007859  1345 raft_consensus.cc:2468] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:59.008407  1419 raft_consensus.cc:2468] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 0.
I20260709 12:21:59.009286  1461 leader_election.cc:304] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:59.007835  1386 leader_election.cc:304] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:59.009899  1340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
I20260709 12:21:59.010643  2022 raft_consensus.cc:2804] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:59.011113  2022 raft_consensus.cc:493] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.011519  2022 raft_consensus.cc:3060] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.010842  1340 raft_consensus.cc:2468] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 0.
I20260709 12:21:59.017516  2057 raft_consensus.cc:2804] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:59.018002  2057 raft_consensus.cc:493] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.018383  2057 raft_consensus.cc:3060] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.048434  2057 raft_consensus.cc:515] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:59.052067  2057 leader_election.cc:290] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:59.053256  2022 raft_consensus.cc:515] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.055420  1622 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:59.054992  1240 catalog_manager.cc:5697] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.056042  1622 ts_tablet_manager.cc:1403] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.024s
I20260709 12:21:59.056833  2022 leader_election.cc:290] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:21:59.057125  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:21:59.057451  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:59.057782  1340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:59.057801  1568 raft_consensus.cc:3060] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.058180  1345 raft_consensus.cc:3060] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.058627  1340 raft_consensus.cc:3060] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.063001  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:59.063642  1419 raft_consensus.cc:3060] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.066551  1568 raft_consensus.cc:2468] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:59.067941  1386 leader_election.cc:304] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:21:59.068864  2074 raft_consensus.cc:2804] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:59.069487  2074 raft_consensus.cc:697] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:21:59.070434  2074 consensus_queue.cc:237] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.076742  1419 raft_consensus.cc:2468] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:59.078148  1463 leader_election.cc:304] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:21:59.078950  1846 raft_consensus.cc:2804] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:59.079631  1846 raft_consensus.cc:697] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:21:59.080439  1846 consensus_queue.cc:237] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:59.093503  2025 raft_consensus.cc:493] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.094058  2025 raft_consensus.cc:515] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.098436  2025 leader_election.cc:290] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:59.099665  1346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c" is_pre_election: true
W20260709 12:21:59.101356  1292 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:59.101589  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:59.102356  1419 raft_consensus.cc:2468] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 0.
W20260709 12:21:59.104212  1535 leader_election.cc:343] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:59.104349  1340 raft_consensus.cc:2468] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:21:59.108268  1537 leader_election.cc:304] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:59.108484  2033 consensus_queue.cc:1048] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.108542  1345 raft_consensus.cc:2468] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:21:59.108914  1622 raft_consensus.cc:359] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.109536  1670 raft_consensus.cc:2804] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:59.109746  1622 raft_consensus.cc:385] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.110230  1622 raft_consensus.cc:740] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.110051  1670 raft_consensus.cc:493] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.110987  1622 consensus_queue.cc:260] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.115346  1670 raft_consensus.cc:3060] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.122678  2008 raft_consensus.cc:493] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.123231  2008 raft_consensus.cc:515] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.128690  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:59.129558  1493 raft_consensus.cc:2393] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
I20260709 12:21:59.131866  1240 catalog_manager.cc:5697] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.132663  2023 consensus_queue.cc:1048] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.142529  1344 raft_consensus.cc:3060] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.147715  1622 ts_tablet_manager.cc:1434] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.091s	user 0.010s	sys 0.005s
I20260709 12:21:59.153965  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:59.155992  1310 leader_election.cc:304] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:59.157331  1866 raft_consensus.cc:3060] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.157940  1622 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:59.164680  2008 leader_election.cc:290] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:59.171226  1622 tablet_bootstrap.cc:654] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:59.178524  1670 raft_consensus.cc:515] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.184895  1866 raft_consensus.cc:2749] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:59.188841  1535 consensus_peers.cc:597] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:59.202802  1345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f9c1c4921946298df299815121637c"
I20260709 12:21:59.194058  1622 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:59.197423  1670 leader_election.cc:290] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:59.228350  1622 ts_tablet_manager.cc:1403] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.011s
W20260709 12:21:59.230032  1535 leader_election.cc:343] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:59.204046  1244 catalog_manager.cc:5697] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 0 to 1, leader changed from <none> to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 1 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.231113  1622 raft_consensus.cc:359] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.231786  1622 raft_consensus.cc:385] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.232059  1622 raft_consensus.cc:740] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.232699  1622 consensus_queue.cc:260] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.234833  1622 ts_tablet_manager.cc:1434] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:59.235724  1622 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:59.189293  1846 consensus_queue.cc:1048] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.247994  1622 tablet_bootstrap.cc:654] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:59.210240  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:21:59.253100  1419 raft_consensus.cc:3060] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.278475  1419 raft_consensus.cc:2468] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:21:59.281051  1537 leader_election.cc:304] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:21:59.281538  1622 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:59.282367  1622 ts_tablet_manager.cc:1403] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.047s	user 0.026s	sys 0.013s
I20260709 12:21:59.285470  2025 raft_consensus.cc:2804] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:59.286059  2025 raft_consensus.cc:697] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:21:59.286947  2025 consensus_queue.cc:237] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.288203  1622 raft_consensus.cc:359] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:59.288964  1622 raft_consensus.cc:385] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.289238  1622 raft_consensus.cc:740] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.289960  1622 consensus_queue.cc:260] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:21:59.296718  1622 ts_tablet_manager.cc:1434] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:59.297768  1622 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:59.308696  1622 tablet_bootstrap.cc:654] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:59.312752  1241 catalog_manager.cc:5697] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 0 to 1, leader changed from <none> to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 1 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.314641  1622 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:59.315121  1622 ts_tablet_manager.cc:1403] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.002s
I20260709 12:21:59.317821  1622 raft_consensus.cc:359] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.318454  1622 raft_consensus.cc:385] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.318737  1622 raft_consensus.cc:740] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.318868  2040 consensus_queue.cc:1048] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.319464  1622 consensus_queue.cc:260] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.321171  2101 raft_consensus.cc:493] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.321643  2101 raft_consensus.cc:515] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.322031  1622 ts_tablet_manager.cc:1434] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:59.323144  1622 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:21:59.325372  2101 leader_election.cc:290] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:21:59.326820  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:21:59.328727  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:59.329006  1622 tablet_bootstrap.cc:654] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:59.334512  1312 leader_election.cc:304] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:21:59.335212  2101 raft_consensus.cc:2749] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:59.338927  1622 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: No bootstrap required, opened a new log
I20260709 12:21:59.339799  1622 ts_tablet_manager.cc:1403] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 12:21:59.342912  1622 raft_consensus.cc:359] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.345490  1622 raft_consensus.cc:385] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.345748  1622 raft_consensus.cc:740] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.348536  1622 consensus_queue.cc:260] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:21:59.351092  1622 ts_tablet_manager.cc:1434] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:59.354843  1346 raft_consensus.cc:3060] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.361616  2025 consensus_queue.cc:1048] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.376180  1345 raft_consensus.cc:3060] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.382277  2049 consensus_queue.cc:1048] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.387003  2033 consensus_queue.cc:1048] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.394455  2033 consensus_queue.cc:1048] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.420528  2033 consensus_queue.cc:1048] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.469518  2049 consensus_queue.cc:1048] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.486809  2057 consensus_queue.cc:1048] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.589376  2022 consensus_queue.cc:1048] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.591715  2101 raft_consensus.cc:493] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.592353  2101 raft_consensus.cc:515] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:21:59.595440  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:21:59.596233  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58e96847ffff43ca9df16b0fb7a718a9" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:21:59.597167  1420 raft_consensus.cc:2393] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f9c1c4921946298df299815121637c in current term 1: Already voted for candidate 76217635dae3410b96baf380d386cbee in this term.
I20260709 12:21:59.597750  2101 leader_election.cc:290] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:21:59.598989  1310 leader_election.cc:304] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d
I20260709 12:21:59.600674  1866 raft_consensus.cc:3060] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.611090  1345 raft_consensus.cc:3060] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.618595  2121 consensus_queue.cc:1048] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.620689  1866 raft_consensus.cc:2749] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:59.667486  1947 consensus_queue.cc:1048] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.680258  2025 consensus_queue.cc:1048] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:00.693362 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
W20260709 12:22:00.698756  1535 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:22:00.703014  1535 consensus_peers.cc:597] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.721376  1535 consensus_peers.cc:597] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.728199  1386 consensus_peers.cc:597] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.734721  1461 consensus_peers.cc:597] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.736647  1461 consensus_peers.cc:597] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.738049  1535 consensus_peers.cc:597] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.743782 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:00.744787 24421 tablet_replica.cc:333] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.745868 24421 raft_consensus.cc:2243] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.746520 24421 raft_consensus.cc:2272] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.748651 24421 tablet_replica.cc:333] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.749467 24421 raft_consensus.cc:2243] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.749965 24421 raft_consensus.cc:2272] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.752472 24421 tablet_replica.cc:333] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.753199 24421 raft_consensus.cc:2243] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.753638 24421 raft_consensus.cc:2272] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.755669 24421 tablet_replica.cc:333] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.756460 24421 raft_consensus.cc:2243] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.756932 24421 raft_consensus.cc:2272] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.757035  1461 consensus_peers.cc:597] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.759060 24421 tablet_replica.cc:333] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.759629 24421 raft_consensus.cc:2243] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.759994 24421 raft_consensus.cc:2272] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.762151 24421 tablet_replica.cc:333] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.762853 24421 raft_consensus.cc:2243] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.763280 24421 raft_consensus.cc:2272] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.765237 24421 tablet_replica.cc:333] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.765980 24421 raft_consensus.cc:2243] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.766391 24421 raft_consensus.cc:2272] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.768445  1461 consensus_peers.cc:597] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.768492 24421 tablet_replica.cc:333] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.769445 24421 raft_consensus.cc:2243] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.769871 24421 raft_consensus.cc:2272] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.771723 24421 tablet_replica.cc:333] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.772315 24421 raft_consensus.cc:2243] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.772714 24421 raft_consensus.cc:2272] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.774827 24421 tablet_replica.cc:333] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.775403 24421 raft_consensus.cc:2243] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.775844 24421 raft_consensus.cc:2272] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.777916 24421 tablet_replica.cc:333] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.778746 24421 raft_consensus.cc:2243] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.779327 24421 raft_consensus.cc:2272] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.781607 24421 tablet_replica.cc:333] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.782397 24421 raft_consensus.cc:2243] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.795008 24421 raft_consensus.cc:2272] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.797420 24421 tablet_replica.cc:333] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: stopping tablet replica
W20260709 12:22:00.797950  1535 consensus_peers.cc:597] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.798152 24421 raft_consensus.cc:2243] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.798856 24421 raft_consensus.cc:2272] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.801163 24421 tablet_replica.cc:333] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.801862 24421 raft_consensus.cc:2243] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.802230 24421 raft_consensus.cc:2272] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.803895 24421 tablet_replica.cc:333] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.804404 24421 raft_consensus.cc:2243] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.804773 24421 raft_consensus.cc:2272] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.807075 24421 tablet_replica.cc:333] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.807708 24421 raft_consensus.cc:2243] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:00.808650 24421 raft_consensus.cc:2272] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.811240 24421 tablet_replica.cc:333] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.812016 24421 raft_consensus.cc:2243] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.812492 24421 raft_consensus.cc:2272] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.812588  1535 consensus_peers.cc:597] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.814190 24421 tablet_replica.cc:333] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.814926 24421 raft_consensus.cc:2243] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.815379 24421 raft_consensus.cc:2272] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.817559 24421 tablet_replica.cc:333] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.818317 24421 raft_consensus.cc:2243] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.818912 24421 raft_consensus.cc:2272] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.821563 24421 tablet_replica.cc:333] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.822331 24421 raft_consensus.cc:2243] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.822866 24421 raft_consensus.cc:2272] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.824424  1386 consensus_peers.cc:597] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.825044 24421 tablet_replica.cc:333] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.825747 24421 raft_consensus.cc:2243] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.826180 24421 raft_consensus.cc:2272] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.828301 24421 tablet_replica.cc:333] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.828868 24421 raft_consensus.cc:2243] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.829274 24421 raft_consensus.cc:2272] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.831174 24421 tablet_replica.cc:333] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.831795 24421 raft_consensus.cc:2243] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.832222 24421 raft_consensus.cc:2272] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.834108 24421 tablet_replica.cc:333] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.834784 24421 raft_consensus.cc:2243] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.835206 24421 raft_consensus.cc:2272] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.837433 24421 tablet_replica.cc:333] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: stopping tablet replica
W20260709 12:22:00.837618  1461 consensus_peers.cc:597] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.838256 24421 raft_consensus.cc:2243] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.838868 24421 raft_consensus.cc:2272] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.841022 24421 tablet_replica.cc:333] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.841647 24421 raft_consensus.cc:2243] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.842072 24421 raft_consensus.cc:2272] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.843775 24421 tablet_replica.cc:333] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.844322 24421 raft_consensus.cc:2243] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.844720 24421 raft_consensus.cc:2272] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.846511 24421 tablet_replica.cc:333] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.847074 24421 raft_consensus.cc:2243] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.847477 24421 raft_consensus.cc:2272] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.849365 24421 tablet_replica.cc:333] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.849926 24421 raft_consensus.cc:2243] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:00.850754 24421 raft_consensus.cc:2272] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.852607 24421 tablet_replica.cc:333] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.853238 24421 raft_consensus.cc:2243] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.853699 24421 raft_consensus.cc:2272] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.855695 24421 tablet_replica.cc:333] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.856240 24421 raft_consensus.cc:2243] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.856684 24421 raft_consensus.cc:2272] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.857261  1386 consensus_peers.cc:597] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.858649 24421 tablet_replica.cc:333] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.859480 24421 raft_consensus.cc:2243] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.859992 24421 raft_consensus.cc:2272] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.861493  1386 consensus_peers.cc:597] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.862448 24421 tablet_replica.cc:333] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.863162 24421 raft_consensus.cc:2243] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.863641 24421 raft_consensus.cc:2272] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.865834 24421 tablet_replica.cc:333] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.866391 24421 raft_consensus.cc:2243] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.866923 24421 raft_consensus.cc:2272] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.869529 24421 tablet_replica.cc:333] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.870306 24421 raft_consensus.cc:2243] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:00.871236 24421 raft_consensus.cc:2272] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.871752  1535 consensus_peers.cc:597] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.873569 24421 tablet_replica.cc:333] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.874400 24421 raft_consensus.cc:2243] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.874897 24421 raft_consensus.cc:2272] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.877302 24421 tablet_replica.cc:333] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.878042 24421 raft_consensus.cc:2243] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.878643 24421 raft_consensus.cc:2272] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.880780 24421 tablet_replica.cc:333] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.881348 24421 raft_consensus.cc:2243] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:00.882144 24421 raft_consensus.cc:2272] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.884124 24421 tablet_replica.cc:333] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.884811 24421 raft_consensus.cc:2243] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.885298 24421 raft_consensus.cc:2272] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.887794 24421 tablet_replica.cc:333] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.888504 24421 raft_consensus.cc:2243] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.889008 24421 raft_consensus.cc:2272] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.890419  1461 consensus_peers.cc:597] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.891235  1461 consensus_peers.cc:597] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.891364 24421 tablet_replica.cc:333] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.892170 24421 raft_consensus.cc:2243] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:00.893075 24421 raft_consensus.cc:2272] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.895480 24421 tablet_replica.cc:333] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.896173 24421 raft_consensus.cc:2243] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.896644 24421 raft_consensus.cc:2272] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.899159 24421 tablet_replica.cc:333] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.899804 24421 raft_consensus.cc:2243] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:00.900708 24421 raft_consensus.cc:2272] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.903182  1461 consensus_peers.cc:597] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.903395 24421 tablet_replica.cc:333] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.904328 24421 raft_consensus.cc:2243] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.904773 24421 raft_consensus.cc:2272] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:00.906828 24421 tablet_replica.cc:333] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: stopping tablet replica
I20260709 12:22:00.907342 24421 raft_consensus.cc:2243] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:00.907785 24421 raft_consensus.cc:2272] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:00.910843  1535 consensus_peers.cc:597] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.921456  1461 consensus_peers.cc:597] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.937144  1535 consensus_peers.cc:597] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.937770  1386 consensus_peers.cc:597] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.970144  1461 consensus_peers.cc:597] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.979260  1461 consensus_peers.cc:597] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.980175 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
W20260709 12:22:01.010955  1386 consensus_peers.cc:597] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.028393  1461 consensus_peers.cc:597] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.039685  1386 consensus_peers.cc:597] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.056692 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:01.061252  1535 consensus_peers.cc:597] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.067312  2132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:01.068477  1461 consensus_peers.cc:597] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.069669  2133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:01.072688 24421 server_base.cc:1061] running on GCE node
W20260709 12:22:01.073086  2135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:01.076133 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:01.076385 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:01.076615 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599721076594 us; error 0 us; skew 500 ppm
I20260709 12:22:01.077236 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:01.080761 24421 webserver.cc:533] Webserver started at http://127.23.217.65:40581/ using document root <none> and password file <none>
I20260709 12:22:01.081373 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:01.081583 24421 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:22:01.086014 24421 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.001s
W20260709 12:22:01.094417  1386 consensus_peers.cc:597] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.099087  2140 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:01.100291 24421 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.002s
I20260709 12:22:01.100734 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "83f9c1c4921946298df299815121637c"
format_stamp: "Formatted at 2026-07-09 12:21:47 on dist-test-slave-lg8f"
W20260709 12:22:01.101012  1535 consensus_peers.cc:597] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.101402 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:22:01.109723  1535 consensus_peers.cc:597] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.117141 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:01.118745 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:01.120558 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260709 12:22:01.128053  1386 consensus_peers.cc:597] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.128019  2148 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:22:01.174973  1535 consensus_peers.cc:597] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.188089  1535 consensus_peers.cc:597] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.199769  1535 consensus_peers.cc:597] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.200248  1535 consensus_peers.cc:597] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.200645  1535 consensus_peers.cc:597] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.226516  1461 consensus_peers.cc:597] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.466974 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:22:01.467392 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.341s	user 0.006s	sys 0.003s
I20260709 12:22:01.467727 24421 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:22:01.474005  2148 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.505838  2148 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.506815  2148 tablet_bootstrap.cc:492] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.507395  2148 ts_tablet_manager.cc:1403] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.012s
I20260709 12:22:01.509687  2148 raft_consensus.cc:359] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.510352  2148 raft_consensus.cc:740] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.511135  2148 consensus_queue.cc:260] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.517268  2148 ts_tablet_manager.cc:1434] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:22:01.518704  2148 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.554068  2148 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.555094  2148 tablet_bootstrap.cc:492] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.555763  2148 ts_tablet_manager.cc:1403] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.004s
I20260709 12:22:01.558797  2148 raft_consensus.cc:359] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.559499  2148 raft_consensus.cc:740] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.560110  2148 consensus_queue.cc:260] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.561589  2148 ts_tablet_manager.cc:1434] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:01.562263  2148 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.575127  2148 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.575989  2148 tablet_bootstrap.cc:492] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.576589  2148 ts_tablet_manager.cc:1403] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 12:22:01.579226  2148 raft_consensus.cc:359] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.579941  2148 raft_consensus.cc:740] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.580732  2148 consensus_queue.cc:260] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.582454  2148 ts_tablet_manager.cc:1434] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:22:01.584383  2148 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.605036  2148 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.606014  2148 tablet_bootstrap.cc:492] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.606653  2148 ts_tablet_manager.cc:1403] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.001s
I20260709 12:22:01.610957  2148 raft_consensus.cc:359] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.611735  2148 raft_consensus.cc:740] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.614348  2148 consensus_queue.cc:260] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.616326  2148 ts_tablet_manager.cc:1434] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:22:01.617290  2148 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.642057  2148 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.642995  2148 tablet_bootstrap.cc:492] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.643602  2148 ts_tablet_manager.cc:1403] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.016s
I20260709 12:22:01.646001  2148 raft_consensus.cc:359] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.646687  2148 raft_consensus.cc:740] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.647536  2148 consensus_queue.cc:260] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.649616  2148 ts_tablet_manager.cc:1434] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:01.650362  2148 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.682564  2148 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.683488  2148 tablet_bootstrap.cc:492] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.684077  2148 ts_tablet_manager.cc:1403] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.004s
I20260709 12:22:01.692845  2148 raft_consensus.cc:359] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.693547  2148 raft_consensus.cc:740] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.694219  2148 consensus_queue.cc:260] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.696086  2148 ts_tablet_manager.cc:1434] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:01.696853  2148 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.729820 24421 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:22:01.730189 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.262s	user 0.243s	sys 0.000s
I20260709 12:22:01.730715  2148 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.732244  2148 tablet_bootstrap.cc:492] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.733088  2148 ts_tablet_manager.cc:1403] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.012s
I20260709 12:22:01.735538  2148 raft_consensus.cc:359] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:01.736263  2148 raft_consensus.cc:740] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.737015  2148 consensus_queue.cc:260] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:01.739015  2148 ts_tablet_manager.cc:1434] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:01.739845  2148 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.812922  2148 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.814069  2148 tablet_bootstrap.cc:492] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.814775  2148 ts_tablet_manager.cc:1403] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.075s	user 0.031s	sys 0.036s
I20260709 12:22:01.817255  2148 raft_consensus.cc:359] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.817835  2148 raft_consensus.cc:740] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.818746  2148 consensus_queue.cc:260] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.821120  2148 ts_tablet_manager.cc:1434] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:01.821960  2148 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.848968  2148 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.850028  2148 tablet_bootstrap.cc:492] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.850899  2148 ts_tablet_manager.cc:1403] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.029s	user 0.022s	sys 0.000s
I20260709 12:22:01.853749  2148 raft_consensus.cc:359] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.854588  2148 raft_consensus.cc:740] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.855340  2148 consensus_queue.cc:260] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.858247  2148 ts_tablet_manager.cc:1434] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:01.859195  2148 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.907821  2147 raft_consensus.cc:493] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:01.908601  2147 raft_consensus.cc:515] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:01.920930  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:22:01.923736  2148 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.924865  2148 tablet_bootstrap.cc:492] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.925577  2148 ts_tablet_manager.cc:1403] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.000s
I20260709 12:22:01.928305  2148 raft_consensus.cc:359] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:01.929098  2148 raft_consensus.cc:740] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.929962  2148 consensus_queue.cc:260] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:01.932559  2148 ts_tablet_manager.cc:1434] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.003s
I20260709 12:22:01.933429  2148 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:01.944953  2147 leader_election.cc:290] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:01.946345  2023 raft_consensus.cc:493] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:01.947090  2023 raft_consensus.cc:515] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:01.949395  2023 leader_election.cc:290] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:01.949544  2057 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:01.953088  2057 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:01.955274  2057 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:01.952324  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:22:01.955956  2148 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.956001  1419 raft_consensus.cc:2468] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:22:01.957731  2148 tablet_bootstrap.cc:492] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:01.958777  2148 ts_tablet_manager.cc:1403] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.004s
I20260709 12:22:01.962214  2148 raft_consensus.cc:359] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.963210  2148 raft_consensus.cc:740] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.964148  2148 consensus_queue.cc:260] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:01.966123  1463 leader_election.cc:304] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:01.966885  2148 ts_tablet_manager.cc:1434] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:01.967043  2023 raft_consensus.cc:2804] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:01.967657  2023 raft_consensus.cc:493] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:01.968001  2023 raft_consensus.cc:3060] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:01.968585  2148 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:22:01.975176  1386 leader_election.cc:336] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:01.978513  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:22:01.981575  1386 leader_election.cc:304] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:22:01.988184  2147 raft_consensus.cc:2749] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:01.989171  2200 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:01.989754  2200 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.001068  2025 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.001116  2148 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.002177  2200 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.002795  2148 tablet_bootstrap.cc:492] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.003990  2148 ts_tablet_manager.cc:1403] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.036s	user 0.026s	sys 0.003s
I20260709 12:22:02.001865  2025 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.006883  2025 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.007963  2148 raft_consensus.cc:359] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.009063  2148 raft_consensus.cc:740] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.010183  2148 consensus_queue.cc:260] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.011044  2023 raft_consensus.cc:515] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.013309  2148 ts_tablet_manager.cc:1434] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:02.014138  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8576156f1c0b4a5288eb4f9501386641" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:02.014549  2148 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.014868  1419 raft_consensus.cc:3060] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.024475  1419 raft_consensus.cc:2468] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 2.
I20260709 12:22:02.024596  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:22:02.025749  1463 leader_election.cc:304] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:02.026638  2200 raft_consensus.cc:2804] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:02.027891  2200 raft_consensus.cc:697] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 2 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:02.029006  2200 consensus_queue.cc:237] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:22:02.028621  1535 leader_election.cc:336] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.030822  2023 leader_election.cc:290] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
W20260709 12:22:02.035681  1461 leader_election.cc:336] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
W20260709 12:22:02.037325  1461 leader_election.cc:336] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.038637  1461 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:22:02.039625  2023 raft_consensus.cc:2749] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:02.042801  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:22:02.043697  1494 raft_consensus.cc:2468] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:02.046700  2147 raft_consensus.cc:493] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.047297  2147 raft_consensus.cc:515] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.049527  2147 leader_election.cc:290] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:02.049407  2148 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.052698  2148 tablet_bootstrap.cc:492] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.053913  2148 ts_tablet_manager.cc:1403] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.008s
I20260709 12:22:02.057704  2148 raft_consensus.cc:359] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.058663  2148 raft_consensus.cc:740] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.058916  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:22:02.060380  1386 leader_election.cc:336] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.063278  2023 raft_consensus.cc:493] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.063660  2148 consensus_queue.cc:260] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.064214  1537 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.065630  2025 raft_consensus.cc:2804] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:02.066018  2025 raft_consensus.cc:493] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.066351  2025 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:02.067162  2148 ts_tablet_manager.cc:1434] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:22:02.064414  1388 leader_election.cc:304] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:22:02.068364  2148 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.063812  2023 raft_consensus.cc:515] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.070824  2023 leader_election.cc:290] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:02.068450  2147 raft_consensus.cc:2749] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.072435  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:22:02.073330  1420 raft_consensus.cc:2468] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 1.
I20260709 12:22:02.072968  1240 catalog_manager.cc:5697] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 1 to 2, leader changed from 83f9c1c4921946298df299815121637c (127.23.217.65) to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:02.075266  1463 leader_election.cc:304] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:02.076929  2200 raft_consensus.cc:2804] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:02.079787  2200 raft_consensus.cc:493] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.080248  2200 raft_consensus.cc:3060] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.082312  2025 raft_consensus.cc:515] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.084539  2025 leader_election.cc:290] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
W20260709 12:22:02.091003  1461 leader_election.cc:336] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.091742  1461 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8246e0558124e98a58932ad7f4676e3; no voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c
I20260709 12:22:02.092520  2023 raft_consensus.cc:2749] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:02.093724  1461 leader_election.cc:336] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.088914  2149 raft_consensus.cc:493] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.095141  2149 raft_consensus.cc:515] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.098610  2149 leader_election.cc:290] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:02.100327  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:22:02.104491  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3da69d29cbf04086b598c879439a9e8d" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:02.105377  1493 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:02.108597  2200 raft_consensus.cc:515] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:22:02.112175  1535 leader_election.cc:336] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.112713  1535 leader_election.cc:304] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:22:02.113502  2025 raft_consensus.cc:2749] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:02.113611  1535 leader_election.cc:336] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.125696  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a040bff9d840498d2e8f9bf328b13b" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:02.126318  1420 raft_consensus.cc:3060] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:02.129302  1461 leader_election.cc:336] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
W20260709 12:22:02.140165  1461 leader_election.cc:336] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.140890  2148 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.134094  1420 raft_consensus.cc:2468] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 2.
I20260709 12:22:02.128242  2200 leader_election.cc:290] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 election: Requested vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:02.143280  1463 leader_election.cc:304] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.143857  2148 tablet_bootstrap.cc:492] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.145205  2148 ts_tablet_manager.cc:1403] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.077s	user 0.019s	sys 0.007s
I20260709 12:22:02.147517  1493 raft_consensus.cc:2468] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 3.
I20260709 12:22:02.148775  2200 raft_consensus.cc:2804] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:02.148263  2148 raft_consensus.cc:359] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.149374  1537 leader_election.cc:304] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.149475  2200 raft_consensus.cc:697] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 2 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:02.150015  2148 raft_consensus.cc:740] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.150919  2149 raft_consensus.cc:2804] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:02.150847  2200 consensus_queue.cc:237] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.151823  2149 raft_consensus.cc:697] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 3 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:02.150995  2148 consensus_queue.cc:260] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.152735  2149 consensus_queue.cc:237] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.154919  2148 ts_tablet_manager.cc:1434] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:22:02.156155  2148 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.177086  2150 raft_consensus.cc:493] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.202139  2150 raft_consensus.cc:515] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.200300  2148 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.204962  2150 leader_election.cc:290] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.207290  2148 tablet_bootstrap.cc:492] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.209067  2148 ts_tablet_manager.cc:1403] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.003s
I20260709 12:22:02.209748  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:22:02.210659  1569 raft_consensus.cc:2468] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:22:02.207777  1670 raft_consensus.cc:493] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
W20260709 12:22:02.211403  1386 leader_election.cc:336] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.211789  1670 raft_consensus.cc:515] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.212723  1386 leader_election.cc:304] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.213760  2150 raft_consensus.cc:2804] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:02.214085  1670 leader_election.cc:290] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:02.214148  2150 raft_consensus.cc:493] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.214951  2150 raft_consensus.cc:3060] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.216899  2148 raft_consensus.cc:359] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.219805  2148 raft_consensus.cc:740] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.221937  2148 consensus_queue.cc:260] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.225732  2150 raft_consensus.cc:515] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.226428  2148 ts_tablet_manager.cc:1434] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:22:02.228647  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e617f860b45428ba84e2c8b7511ae75" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:02.230916  1569 raft_consensus.cc:3060] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.231348  2148 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:22:02.231724  1535 leader_election.cc:336] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
W20260709 12:22:02.238368  1386 leader_election.cc:336] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.239593  2150 leader_election.cc:290] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.240456  1569 raft_consensus.cc:2468] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:02.241796  1420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "95782952bdee421094cada4ea955015d" is_pre_election: true
I20260709 12:22:02.241923  1386 leader_election.cc:304] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.244243  2150 raft_consensus.cc:2804] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:02.245437  2150 raft_consensus.cc:697] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:02.246284  2150 consensus_queue.cc:237] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.251989  1537 leader_election.cc:304] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:22:02.253357  2033 raft_consensus.cc:2749] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.269800  2148 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.276315  2148 tablet_bootstrap.cc:492] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.277154  2148 ts_tablet_manager.cc:1403] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.003s
I20260709 12:22:02.283111  2148 raft_consensus.cc:359] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.283860  2148 raft_consensus.cc:740] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.287835  2148 consensus_queue.cc:260] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.292462  2148 ts_tablet_manager.cc:1434] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:22:02.293550  2148 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.337071  2148 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.345697  1244 catalog_manager.cc:5697] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 2 to 3, leader changed from 83f9c1c4921946298df299815121637c (127.23.217.65) to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 3 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:02.338371  2148 tablet_bootstrap.cc:492] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.360707  2148 ts_tablet_manager.cc:1403] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.006s
I20260709 12:22:02.363943  2148 raft_consensus.cc:359] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.364674  2148 raft_consensus.cc:740] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.365381  2148 consensus_queue.cc:260] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.367758  2148 ts_tablet_manager.cc:1434] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:22:02.368630  2148 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.371575  2149 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.372252  2149 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.378360  2121 raft_consensus.cc:493] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.378997  2121 raft_consensus.cc:515] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.380944  2121 leader_election.cc:290] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.389101  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:22:02.392460  1568 raft_consensus.cc:2468] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 1.
I20260709 12:22:02.374369  2149 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.389716  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
W20260709 12:22:02.390676  1535 leader_election.cc:336] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.393739  1495 raft_consensus.cc:2468] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 1.
I20260709 12:22:02.396687  1537 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.397163  1386 leader_election.cc:304] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:02.381946  1240 catalog_manager.cc:5697] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 1 to 2, leader changed from 83f9c1c4921946298df299815121637c (127.23.217.65) to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:02.397440  2025 raft_consensus.cc:2804] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:02.397874  2025 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.398190  2025 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.404259  2148 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.405390  2148 tablet_bootstrap.cc:492] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.406121  2148 ts_tablet_manager.cc:1403] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.004s
I20260709 12:22:02.407356  2121 raft_consensus.cc:2804] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:02.407873  2121 raft_consensus.cc:493] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:02.408262  2121 raft_consensus.cc:3060] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.408861  2148 raft_consensus.cc:359] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.409626  2148 raft_consensus.cc:740] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
W20260709 12:22:02.412994  1386 leader_election.cc:336] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.414767  2148 consensus_queue.cc:260] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.418998  2148 ts_tablet_manager.cc:1434] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:22:02.419888  2148 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.431128  1240 catalog_manager.cc:5697] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 1 to 2, leader changed from 83f9c1c4921946298df299815121637c (127.23.217.65) to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 2 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:02.433916  2121 raft_consensus.cc:515] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.438361  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3f7b195a584dd1896512e1497e4ead" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:02.439041  1568 raft_consensus.cc:3060] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.437374  2025 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.456773  2025 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.462335  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:02.463130  1495 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:02.469830  1386 leader_election.cc:336] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.473037  1568 raft_consensus.cc:2468] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
W20260709 12:22:02.470132  1535 leader_election.cc:336] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:02.480967  1386 leader_election.cc:304] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.482179  2121 leader_election.cc:290] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.482659  2147 raft_consensus.cc:2804] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:02.483276  2147 raft_consensus.cc:697] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:02.484707  2147 consensus_queue.cc:237] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:02.483115  1495 raft_consensus.cc:2468] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:02.492244  1537 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3; no voters: 83f9c1c4921946298df299815121637c
I20260709 12:22:02.493312  2149 raft_consensus.cc:2804] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:02.493826  2149 raft_consensus.cc:697] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:02.494675  2149 consensus_queue.cc:237] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.507473  1240 catalog_manager.cc:5697] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 1 to 2, leader changed from 83f9c1c4921946298df299815121637c (127.23.217.65) to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 2 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } }
W20260709 12:22:02.520509  1463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 53824.4us
I20260709 12:22:02.536530  1244 catalog_manager.cc:5697] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 1 to 2, leader changed from 83f9c1c4921946298df299815121637c (127.23.217.65) to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 2 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:02.539057  2148 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.540534  2148 tablet_bootstrap.cc:492] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.541499  2148 ts_tablet_manager.cc:1403] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.122s	user 0.024s	sys 0.020s
I20260709 12:22:02.548396 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:37337
I20260709 12:22:02.548636  2240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:37337 every 8 connection(s)
I20260709 12:22:02.549943  2148 raft_consensus.cc:359] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.552464  2148 raft_consensus.cc:740] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.559399  2148 consensus_queue.cc:260] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.567029  2148 ts_tablet_manager.cc:1434] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.000s
I20260709 12:22:02.568897  2148 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.618456  2193 raft_consensus.cc:3060] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.631279  1417 raft_consensus.cc:1275] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer e8246e0558124e98a58932ad7f4676e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.632920  2237 consensus_queue.cc:1048] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:02.634454  1461 consensus_peers.cc:597] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:02.647825  1422 raft_consensus.cc:1275] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer e8246e0558124e98a58932ad7f4676e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.649318  2237 consensus_queue.cc:1048] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:02.663759  2193 raft_consensus.cc:1275] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Refusing update from remote peer e8246e0558124e98a58932ad7f4676e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.670991  2057 consensus_queue.cc:1048] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:02.703460  1568 raft_consensus.cc:1275] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.706028  2150 consensus_queue.cc:1048] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.741356  2148 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.742460  2148 tablet_bootstrap.cc:492] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.743328  2148 ts_tablet_manager.cc:1403] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.175s	user 0.060s	sys 0.067s
I20260709 12:22:02.746068  2148 raft_consensus.cc:359] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.748998  2148 raft_consensus.cc:740] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.749560  1495 raft_consensus.cc:1275] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:02.749742  2148 consensus_queue.cc:260] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.754455  2148 ts_tablet_manager.cc:1434] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:22:02.755033  2149 consensus_queue.cc:1048] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 12:22:02.772724  1535 consensus_peers.cc:597] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:02.774789  2148 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.777051  2152 raft_consensus.cc:493] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.777740  2152 raft_consensus.cc:515] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:02.778573  2242 heartbeater.cc:344] Connected to a master server at 127.23.217.126:45747
I20260709 12:22:02.779109  2242 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:02.780058  2242 heartbeater.cc:507] Master 127.23.217.126:45747 requested a full tablet report, sending...
I20260709 12:22:02.786860  2152 leader_election.cc:290] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
W20260709 12:22:02.857807  1386 consensus_peers.cc:597] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:02.902969  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cda1629f4344ad8b1ebed68a83d0418" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "76217635dae3410b96baf380d386cbee" is_pre_election: true
I20260709 12:22:02.922516  2191 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.929298  2148 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.930301  2148 tablet_bootstrap.cc:492] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.930161  2191 raft_consensus.cc:1275] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.931324  2148 ts_tablet_manager.cc:1403] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.157s	user 0.034s	sys 0.035s
I20260709 12:22:02.933648  2265 consensus_queue.cc:1048] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:02.934006  2148 raft_consensus.cc:359] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.934741  2148 raft_consensus.cc:740] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.934837  1240 ts_manager.cc:194] Re-registered known tserver with Master: 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
I20260709 12:22:02.935847  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cda1629f4344ad8b1ebed68a83d0418" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8246e0558124e98a58932ad7f4676e3" is_pre_election: true
I20260709 12:22:02.937819  2148 consensus_queue.cc:260] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:02.938649  2143 leader_election.cc:304] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f9c1c4921946298df299815121637c; no voters: 76217635dae3410b96baf380d386cbee, e8246e0558124e98a58932ad7f4676e3
I20260709 12:22:02.939570  2152 raft_consensus.cc:2749] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.942793  2148 ts_tablet_manager.cc:1434] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 12:22:02.943986  2148 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.957274  1494 raft_consensus.cc:1275] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.958717  2149 consensus_queue.cc:1048] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:02.974205  2148 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.975644  2148 tablet_bootstrap.cc:492] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:02.976468  2148 ts_tablet_manager.cc:1403] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.008s
I20260709 12:22:02.979077  2148 raft_consensus.cc:359] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:02.979756  2148 raft_consensus.cc:740] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.980412  2148 consensus_queue.cc:260] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:02.981521  1569 raft_consensus.cc:1275] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:02.983208  2148 ts_tablet_manager.cc:1434] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:02.984081  2148 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:02.984908  2121 consensus_queue.cc:1048] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:03.009533  2148 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.010836  2148 tablet_bootstrap.cc:492] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.011032  2190 raft_consensus.cc:3060] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.011425  2148 ts_tablet_manager.cc:1403] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.008s
I20260709 12:22:03.014480  2148 raft_consensus.cc:359] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.015163  2148 raft_consensus.cc:740] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.015870  2148 consensus_queue.cc:260] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.017838  2148 ts_tablet_manager.cc:1434] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:22:03.021283  2148 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.035472  1240 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55438
I20260709 12:22:03.037025  2190 raft_consensus.cc:1275] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:03.040108  2191 raft_consensus.cc:3060] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.041968  2148 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.054533  2148 tablet_bootstrap.cc:492] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.055181  2148 ts_tablet_manager.cc:1403] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.003s
I20260709 12:22:03.057230  2242 heartbeater.cc:499] Master 127.23.217.126:45747 was elected leader, sending a full tablet report...
I20260709 12:22:03.057847  2148 raft_consensus.cc:359] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.055076  2121 consensus_queue.cc:1048] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:22:03.063434  2148 raft_consensus.cc:740] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.064270  2148 consensus_queue.cc:260] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.071750  2148 ts_tablet_manager.cc:1434] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:22:03.072922  2148 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.135878  2148 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.136701  2148 tablet_bootstrap.cc:492] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.137266  2148 ts_tablet_manager.cc:1403] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.001s
I20260709 12:22:03.139302  2148 raft_consensus.cc:359] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.139854  2148 raft_consensus.cc:740] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.140952  2148 consensus_queue.cc:260] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.150981  2148 ts_tablet_manager.cc:1434] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:22:03.152212  2148 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.166177  2148 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.167106  2148 tablet_bootstrap.cc:492] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.167784  2148 ts_tablet_manager.cc:1403] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:22:03.172855  2148 raft_consensus.cc:359] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.173560  2148 raft_consensus.cc:740] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.174238  2148 consensus_queue.cc:260] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.177441  2148 ts_tablet_manager.cc:1434] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:22:03.178207  2148 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.192977  2148 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.194029  2148 tablet_bootstrap.cc:492] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.194712  2148 ts_tablet_manager.cc:1403] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:22:03.197882  2148 raft_consensus.cc:359] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.198628  2148 raft_consensus.cc:740] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.199302  2148 consensus_queue.cc:260] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.201172  2148 ts_tablet_manager.cc:1434] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:22:03.202026  2148 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.227041  2148 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.228276  2148 tablet_bootstrap.cc:492] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.228979  2148 ts_tablet_manager.cc:1403] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.000s
I20260709 12:22:03.231556  2148 raft_consensus.cc:359] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.232247  2148 raft_consensus.cc:740] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.233035  2148 consensus_queue.cc:260] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.235116  2148 ts_tablet_manager.cc:1434] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:22:03.235877  2148 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:22:03.249472  1461 consensus_peers.cc:597] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.252038  2148 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.253136  2148 tablet_bootstrap.cc:492] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.253798  2148 ts_tablet_manager.cc:1403] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.007s
I20260709 12:22:03.257527  2148 raft_consensus.cc:359] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.258181  2148 raft_consensus.cc:740] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.258886  2148 consensus_queue.cc:260] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.261868  2148 ts_tablet_manager.cc:1434] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:03.262730  2148 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:22:03.274639  1386 consensus_peers.cc:597] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.279047  2148 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.279946  2148 tablet_bootstrap.cc:492] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.280551  2148 ts_tablet_manager.cc:1403] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.001s
I20260709 12:22:03.282465  2148 raft_consensus.cc:359] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.283151  2148 raft_consensus.cc:740] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.283782  2148 consensus_queue.cc:260] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
W20260709 12:22:03.284694  1386 consensus_peers.cc:597] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.285704  2148 ts_tablet_manager.cc:1434] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:22:03.286752  2148 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:22:03.296455  1461 consensus_peers.cc:597] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:03.297336  1461 consensus_peers.cc:597] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:03.297587  1535 consensus_peers.cc:597] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.306249  2148 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.309947  2148 tablet_bootstrap.cc:492] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.310638  2148 ts_tablet_manager.cc:1403] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.011s
I20260709 12:22:03.313062  2148 raft_consensus.cc:359] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.313726  2148 raft_consensus.cc:740] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.314443  2148 consensus_queue.cc:260] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.316394  2148 ts_tablet_manager.cc:1434] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:03.318362  2148 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:22:03.339797  1461 consensus_peers.cc:597] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.343030  2148 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.344153  2148 tablet_bootstrap.cc:492] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.344848  2148 ts_tablet_manager.cc:1403] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.004s
I20260709 12:22:03.347308  2148 raft_consensus.cc:359] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.348022  2148 raft_consensus.cc:740] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.348747  2148 consensus_queue.cc:260] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.350908  2148 ts_tablet_manager.cc:1434] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:03.351713  2148 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.368257  2148 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.369094  2148 tablet_bootstrap.cc:492] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.369812  2148 ts_tablet_manager.cc:1403] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 12:22:03.372351  2148 raft_consensus.cc:359] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.373075  2148 raft_consensus.cc:740] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.374660  2148 consensus_queue.cc:260] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.376884  2148 ts_tablet_manager.cc:1434] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:22:03.378443  2148 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.398972  2148 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.399992  2148 tablet_bootstrap.cc:492] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.400791  2148 ts_tablet_manager.cc:1403] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 12:22:03.403391  2148 raft_consensus.cc:359] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.404327  2148 raft_consensus.cc:740] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.405001  2148 consensus_queue.cc:260] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.407158  2148 ts_tablet_manager.cc:1434] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:03.409405  2148 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.446559  2148 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.453761  2148 tablet_bootstrap.cc:492] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.455482  2148 ts_tablet_manager.cc:1403] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.046s	user 0.018s	sys 0.016s
I20260709 12:22:03.458413  2148 raft_consensus.cc:359] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.459107  2148 raft_consensus.cc:740] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.459962  2148 consensus_queue.cc:260] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.463235  2148 ts_tablet_manager.cc:1434] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:03.465662  2148 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.488035  2148 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.489240  2148 tablet_bootstrap.cc:492] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.489964  2148 ts_tablet_manager.cc:1403] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.002s
I20260709 12:22:03.492297  2148 raft_consensus.cc:359] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.492969  2148 raft_consensus.cc:740] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.493537  2148 consensus_queue.cc:260] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
W20260709 12:22:03.494037  1386 consensus_peers.cc:597] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.496031  2148 ts_tablet_manager.cc:1434] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:22:03.497620  2148 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.515240  2148 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.516456  2148 tablet_bootstrap.cc:492] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.517277  2148 ts_tablet_manager.cc:1403] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 12:22:03.520037  2148 raft_consensus.cc:359] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.520843  2148 raft_consensus.cc:740] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.521541  2148 consensus_queue.cc:260] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
W20260709 12:22:03.522046  1386 consensus_peers.cc:597] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.523947  2148 ts_tablet_manager.cc:1434] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:22:03.525451  2148 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Bootstrap starting.
W20260709 12:22:03.530469  1461 consensus_peers.cc:597] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.549105  2148 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.550283  2148 tablet_bootstrap.cc:492] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.551061  2148 ts_tablet_manager.cc:1403] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.008s
I20260709 12:22:03.553714  2148 raft_consensus.cc:359] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.554423  2148 raft_consensus.cc:740] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.555280  2148 consensus_queue.cc:260] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.558516  2148 ts_tablet_manager.cc:1434] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:03.559303  2148 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.605329  2148 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.606415  2148 tablet_bootstrap.cc:492] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.607175  2148 ts_tablet_manager.cc:1403] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.048s	user 0.023s	sys 0.011s
I20260709 12:22:03.609786  2148 raft_consensus.cc:359] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.610512  2148 raft_consensus.cc:740] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.615835  2148 consensus_queue.cc:260] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.620662  2148 ts_tablet_manager.cc:1434] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:22:03.621476  2148 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.633411  2191 raft_consensus.cc:3060] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.666443  2148 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.667944  2148 tablet_bootstrap.cc:492] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.668869  2148 ts_tablet_manager.cc:1403] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.048s	user 0.030s	sys 0.013s
I20260709 12:22:03.674621  2148 raft_consensus.cc:359] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.675622  2148 raft_consensus.cc:740] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.676770  2148 consensus_queue.cc:260] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:03.679898  2148 ts_tablet_manager.cc:1434] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.002s
I20260709 12:22:03.681057  2148 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.698323  2148 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.699157  2148 tablet_bootstrap.cc:492] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.700256  2148 ts_tablet_manager.cc:1403] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 12:22:03.702400  2148 raft_consensus.cc:359] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.703154  2148 raft_consensus.cc:740] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.705066  2148 consensus_queue.cc:260] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:03.707576  2148 ts_tablet_manager.cc:1434] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:22:03.708449  2148 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.761157  2148 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.762142  2148 tablet_bootstrap.cc:492] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.762840  2148 ts_tablet_manager.cc:1403] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.003s
I20260709 12:22:03.765213  2148 raft_consensus.cc:359] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.765870  2148 raft_consensus.cc:740] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.767514  2148 consensus_queue.cc:260] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:03.771130  2148 ts_tablet_manager.cc:1434] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:22:03.772042  2148 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:03.833947  2148 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.919962  2148 tablet_bootstrap.cc:492] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:03.921593  2148 ts_tablet_manager.cc:1403] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.150s	user 0.011s	sys 0.009s
W20260709 12:22:03.987854  1386 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.68:37405, user_credentials={real_user=slave}} blocked reactor thread for 50496.8us
I20260709 12:22:03.992030  2189 raft_consensus.cc:3060] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.981796  2148 raft_consensus.cc:359] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:04.013437  2148 raft_consensus.cc:740] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.047312  2148 consensus_queue.cc:260] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:04.071893  2148 ts_tablet_manager.cc:1434] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.150s	user 0.011s	sys 0.000s
I20260709 12:22:04.073545  2148 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:04.152992  2148 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.163492  2148 tablet_bootstrap.cc:492] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:04.166612  2148 ts_tablet_manager.cc:1403] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.094s	user 0.027s	sys 0.004s
I20260709 12:22:04.193198  2148 raft_consensus.cc:359] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:04.194022  2148 raft_consensus.cc:740] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.200174  2148 consensus_queue.cc:260] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:04.209574  2148 ts_tablet_manager.cc:1434] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:22:04.212859  2148 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Bootstrap starting.
I20260709 12:22:04.233903  2148 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.234880  2148 tablet_bootstrap.cc:492] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Bootstrap complete.
I20260709 12:22:04.235608  2148 ts_tablet_manager.cc:1403] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 12:22:04.238384  2148 raft_consensus.cc:359] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:04.239140  2148 raft_consensus.cc:740] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f9c1c4921946298df299815121637c, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.239893  2148 consensus_queue.cc:260] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:04.241988  2148 ts_tablet_manager.cc:1434] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:12.756349 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:12.817092  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:12.817716  1569 raft_consensus.cc:606] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:12.820314  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:12.820814  1569 raft_consensus.cc:606] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:12.823589  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:12.824234  1569 raft_consensus.cc:606] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:12.841087  1493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:12.841729  1493 raft_consensus.cc:606] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:12.844945  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b5431f408774554975e15a5998e2062"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:12.845412  1569 raft_consensus.cc:606] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 95782952bdee421094cada4ea955015d
I20260709 12:22:12.848078  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d142957b397e4f75b5526996e950d768"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:12.848583  1569 raft_consensus.cc:606] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:12.849882 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:12.851817 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:12.852638 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:12.879284  2321 raft_consensus.cc:993] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:12.880573  2313 raft_consensus.cc:1081] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:12.882840  2192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42804
I20260709 12:22:12.883440  2192 raft_consensus.cc:493] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:12.883785  2192 raft_consensus.cc:3060] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.893467  2192 raft_consensus.cc:515] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:12.898764  2192 leader_election.cc:290] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:12.899441  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:12.901003  1569 raft_consensus.cc:3055] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:12.901700  1569 raft_consensus.cc:740] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:12.902937  1569 consensus_queue.cc:260] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:12.905373  1569 raft_consensus.cc:3060] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.919931  1569 raft_consensus.cc:2468] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:12.921344  2141 leader_election.cc:304] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:22:12.923494  2329 raft_consensus.cc:2804] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:12.929705  2329 raft_consensus.cc:697] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:12.930398  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce2de400513464ca379c256dc7b0e89" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:12.931273  1419 raft_consensus.cc:3060] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:12.931157  2329 consensus_queue.cc:237] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:12.944883  1419 raft_consensus.cc:2468] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:12.948166  1241 catalog_manager.cc:5697] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:12.972561  2313 raft_consensus.cc:993] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:12.973090  2326 raft_consensus.cc:1081] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:12.974987  2191 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42804
I20260709 12:22:12.975436  2191 raft_consensus.cc:493] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:12.975693  2191 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:12.980906  2191 raft_consensus.cc:515] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:12.982458  2191 leader_election.cc:290] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 3 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:12.983505  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:12.983619  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c81e1068f745e09d8d065e567c4b22" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:12.984231  1493 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:12.984283  1569 raft_consensus.cc:3055] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:12.984748  1569 raft_consensus.cc:740] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:12.985402  1569 consensus_queue.cc:260] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:12.986413  1569 raft_consensus.cc:3060] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:12.991756  1493 raft_consensus.cc:2468] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 3.
I20260709 12:22:12.992218  1569 raft_consensus.cc:2468] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 3.
I20260709 12:22:12.993377  2143 leader_election.cc:304] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:12.994285  2329 raft_consensus.cc:2804] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:12.994758  2329 raft_consensus.cc:697] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 3 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:12.995699  2329 consensus_queue.cc:237] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:13.006035  1241 catalog_manager.cc:5697] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 2 to 3, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 3 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.024186  2049 raft_consensus.cc:993] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:13.024886  2322 raft_consensus.cc:1081] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:13.026856  2191 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42794
I20260709 12:22:13.027655  2191 raft_consensus.cc:493] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.028034  2191 raft_consensus.cc:3060] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.035905  2191 raft_consensus.cc:515] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:13.037503  2191 leader_election.cc:290] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:13.038268  1419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:13.038686  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1b0961fa034e77877efe85062b2307" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:13.038945  1419 raft_consensus.cc:3060] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.039417  1493 raft_consensus.cc:3055] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.039733  1493 raft_consensus.cc:740] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:13.040361  1493 consensus_queue.cc:260] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:13.041296  1493 raft_consensus.cc:3060] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.044901  1419 raft_consensus.cc:2468] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:13.046336  2143 leader_election.cc:304] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:13.047248  2329 raft_consensus.cc:2804] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.047732  2329 raft_consensus.cc:697] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:13.048655  2329 consensus_queue.cc:237] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:13.060458  1241 catalog_manager.cc:5697] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.062347  1493 raft_consensus.cc:2468] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:13.128806  2321 raft_consensus.cc:993] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:13.129439  2313 raft_consensus.cc:1081] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:13.130993  2189 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42804
I20260709 12:22:13.131593  2189 raft_consensus.cc:493] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.131966  2189 raft_consensus.cc:3060] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.139492  2189 raft_consensus.cc:515] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:13.141175  2189 leader_election.cc:290] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:13.141986  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:13.142210  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b2d9d95dad4ef2ae0f868e611b9eab" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:13.142726  1493 raft_consensus.cc:3060] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.142879  1569 raft_consensus.cc:3055] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.143321  1569 raft_consensus.cc:740] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:13.143992  1569 consensus_queue.cc:260] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:13.145143  1569 raft_consensus.cc:3060] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.152927  1569 raft_consensus.cc:2468] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:13.153975  2141 leader_election.cc:304] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:22:13.154832  2329 raft_consensus.cc:2804] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.154698  1493 raft_consensus.cc:2468] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:13.155490  2329 raft_consensus.cc:697] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:13.156533  2329 consensus_queue.cc:237] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:13.170325  1241 catalog_manager.cc:5697] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.226046  2321 raft_consensus.cc:993] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee: : Instructing follower 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:13.226608  2313 raft_consensus.cc:1081] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:13.228572  1417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b5431f408774554975e15a5998e2062"
dest_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:57328
I20260709 12:22:13.229203  1417 raft_consensus.cc:493] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.229516  1417 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.236038  1417 raft_consensus.cc:515] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:13.238572  1417 leader_election.cc:290] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:13.239319  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:13.239801  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:13.240023  1493 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.240473  1569 raft_consensus.cc:3055] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.240783  1569 raft_consensus.cc:740] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:13.241642  1569 consensus_queue.cc:260] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:13.243340  1569 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.247740  1493 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:13.249131  1388 leader_election.cc:304] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:13.250138  2316 raft_consensus.cc:2804] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.250766  2316 raft_consensus.cc:697] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:13.299173  2316 consensus_queue.cc:237] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:13.305161  1569 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:13.320278  1241 catalog_manager.cc:5697] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 2 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.349200  2342 raft_consensus.cc:993] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:13.349706  2321 raft_consensus.cc:1081] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:13.351271  2192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d142957b397e4f75b5526996e950d768"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42804
I20260709 12:22:13.351704  2192 raft_consensus.cc:493] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.351938  2192 raft_consensus.cc:3060] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.357407  2192 raft_consensus.cc:515] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:13.359252  2192 leader_election.cc:290] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:13.359963  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:13.360706  1493 raft_consensus.cc:3060] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.360571  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d142957b397e4f75b5526996e950d768" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:13.361431  1568 raft_consensus.cc:3055] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.361757  1568 raft_consensus.cc:740] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:13.362601  1568 consensus_queue.cc:260] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:13.363976  1568 raft_consensus.cc:3060] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.366561  1493 raft_consensus.cc:2468] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:13.367964  2143 leader_election.cc:304] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:13.368857  2329 raft_consensus.cc:2804] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.369385  2329 raft_consensus.cc:697] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:13.370332  2329 consensus_queue.cc:237] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:13.413400  1569 raft_consensus.cc:1275] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.413909  1417 raft_consensus.cc:1275] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.415125  2329 consensus_queue.cc:1048] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:13.424466  2329 consensus_queue.cc:1048] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.416527  1568 raft_consensus.cc:2468] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:13.426908  1241 catalog_manager.cc:5697] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.449213  1424 raft_consensus.cc:1275] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.451174  2329 consensus_queue.cc:1048] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.486193  1493 raft_consensus.cc:1275] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.491595  2330 consensus_queue.cc:1048] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:22:13.615551  1568 raft_consensus.cc:1275] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:13.617015  2354 consensus_queue.cc:1048] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:13.619887  1493 raft_consensus.cc:1275] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:13.621377  2354 consensus_queue.cc:1048] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:13.762732  1493 raft_consensus.cc:1275] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.764410  2354 consensus_queue.cc:1048] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.772596  1569 raft_consensus.cc:1275] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.774248  2354 consensus_queue.cc:1048] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:13.829030  1569 raft_consensus.cc:1275] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.830344  2340 consensus_queue.cc:1048] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.851009  1493 raft_consensus.cc:1275] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.852581  2340 consensus_queue.cc:1048] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:13.900099 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:13.924243  1568 raft_consensus.cc:1275] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.927415  1493 raft_consensus.cc:1275] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:13.932180  2354 consensus_queue.cc:1048] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:22:13.942582  2330 consensus_queue.cc:1048] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:22:14.045190  1463 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:33559, user_credentials={real_user=slave}} blocked reactor thread for 90843.5us
I20260709 12:22:14.202189  1495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:14.203145  1495 raft_consensus.cc:606] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 95782952bdee421094cada4ea955015d
I20260709 12:22:14.206715  1495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4af17dabf7ac45919d736585909bca81"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:14.207242  1495 raft_consensus.cc:606] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:14.209854  1495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:14.210278  1495 raft_consensus.cc:606] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 76217635dae3410b96baf380d386cbee
I20260709 12:22:14.212725  1495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:14.213230  1495 raft_consensus.cc:606] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 76217635dae3410b96baf380d386cbee
I20260709 12:22:14.216053  1495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:14.216643  1495 raft_consensus.cc:606] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:14.217741 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:14.219063 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:14.219800 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:14.428009  2366 raft_consensus.cc:993] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:14.428666  2366 raft_consensus.cc:1081] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:14.430824  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a"
dest_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:45772
I20260709 12:22:14.431511  1569 raft_consensus.cc:493] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:14.431872  1569 raft_consensus.cc:3060] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.443979  1569 raft_consensus.cc:515] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.447268  1569 leader_election.cc:290] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:14.448201  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:14.449021  1424 raft_consensus.cc:3060] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.450874  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57f5ec0d79454aa37f1c7cd9e30a3a" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:14.451592  1493 raft_consensus.cc:3055] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:14.451874  1493 raft_consensus.cc:740] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:14.452787  1493 consensus_queue.cc:260] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.454113  1493 raft_consensus.cc:3060] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.456089  1424 raft_consensus.cc:2468] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:14.457561  1537 leader_election.cc:304] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:14.458458  2326 raft_consensus.cc:2804] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:14.458953  2326 raft_consensus.cc:697] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:14.460072  2326 consensus_queue.cc:237] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.460616  1493 raft_consensus.cc:2468] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:14.473592  1241 catalog_manager.cc:5697] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 2 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:14.511363  2370 raft_consensus.cc:993] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:14.511966  2366 raft_consensus.cc:1081] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:14.513808  2192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4af17dabf7ac45919d736585909bca81"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42794
I20260709 12:22:14.514405  2192 raft_consensus.cc:493] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:14.514791  2192 raft_consensus.cc:3060] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.522789  2192 raft_consensus.cc:515] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:14.524469  2192 leader_election.cc:290] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:14.525664  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:14.525902  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af17dabf7ac45919d736585909bca81" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:14.526310  1417 raft_consensus.cc:3060] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.526831  1495 raft_consensus.cc:3055] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:14.527220  1495 raft_consensus.cc:740] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:14.528272  1495 consensus_queue.cc:260] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:14.529422  1495 raft_consensus.cc:3060] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.533954  1417 raft_consensus.cc:2468] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:14.535458  2143 leader_election.cc:304] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:14.536481  2330 raft_consensus.cc:2804] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:14.537292  2330 raft_consensus.cc:697] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:14.538395  2330 consensus_queue.cc:237] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:14.541911  1495 raft_consensus.cc:2468] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:14.551038  1241 catalog_manager.cc:5697] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:14.663688  2366 raft_consensus.cc:993] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:14.664562  2379 raft_consensus.cc:1081] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:14.668025  2189 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42794
I20260709 12:22:14.668527  2189 raft_consensus.cc:493] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:14.668815  2189 raft_consensus.cc:3060] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.674578  2189 raft_consensus.cc:515] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:14.675961  2189 leader_election.cc:290] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:14.676903  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:14.677551  1417 raft_consensus.cc:3060] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.677421  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec6fa8077604b94a27cbc5c876af6be" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:14.678442  1493 raft_consensus.cc:3055] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:14.678802  1493 raft_consensus.cc:740] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:14.679647  1493 consensus_queue.cc:260] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:14.680757  1493 raft_consensus.cc:3060] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.685354  1417 raft_consensus.cc:2468] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:14.686782  2143 leader_election.cc:304] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:14.687565  2330 raft_consensus.cc:2804] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:14.688015  2330 raft_consensus.cc:697] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:14.688901  2330 consensus_queue.cc:237] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:14.690100  1493 raft_consensus.cc:2468] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:14.695302  2379 raft_consensus.cc:993] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:14.695977  2367 raft_consensus.cc:1081] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:14.697610  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10"
dest_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:45772
I20260709 12:22:14.698230  1569 raft_consensus.cc:493] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:14.698591  1569 raft_consensus.cc:3060] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.700639  1241 catalog_manager.cc:5697] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:14.707144  1569 raft_consensus.cc:515] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.711272  1569 leader_election.cc:290] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:14.713188  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:14.713810  1417 raft_consensus.cc:3060] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.715317  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d9872c5f8bf40259f4247e381e78e10" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:14.722220  1417 raft_consensus.cc:2468] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:14.716104  1493 raft_consensus.cc:3055] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:14.729085  1493 raft_consensus.cc:740] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:14.729269  1537 leader_election.cc:304] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:14.730062  1493 consensus_queue.cc:260] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.730590  2321 raft_consensus.cc:2804] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:14.731143  2321 raft_consensus.cc:697] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:14.733179  1493 raft_consensus.cc:3060] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.733315  2321 consensus_queue.cc:237] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.739694  1493 raft_consensus.cc:2468] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:14.743050  2370 raft_consensus.cc:993] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:14.743587  2367 raft_consensus.cc:1081] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:14.745754  1417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff"
dest_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:57318
I20260709 12:22:14.746359  1417 raft_consensus.cc:493] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:14.746755  1417 raft_consensus.cc:3060] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.753402  1417 raft_consensus.cc:515] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.755252  1417 leader_election.cc:290] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:14.754467  1241 catalog_manager.cc:5697] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 2 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:14.755895  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:14.756475  1495 raft_consensus.cc:3055] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:14.756770  1495 raft_consensus.cc:740] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:14.757728  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1508cd4f1b544868bbb54eb5a6f4a6ff" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:14.757577  1495 consensus_queue.cc:260] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.758332  1568 raft_consensus.cc:3060] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.758709  1495 raft_consensus.cc:3060] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.766229  1568 raft_consensus.cc:2468] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:14.768306  1386 leader_election.cc:304] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:14.769701  2340 raft_consensus.cc:2804] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:14.770174  2340 raft_consensus.cc:697] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:14.770982  2340 consensus_queue.cc:237] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:14.771342  1495 raft_consensus.cc:2468] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:14.783016  1241 catalog_manager.cc:5697] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 2 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:14.992200  1495 raft_consensus.cc:1275] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.993564  2326 consensus_queue.cc:1048] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.007648  1424 raft_consensus.cc:1275] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.009908  2326 consensus_queue.cc:1048] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.041383  1424 raft_consensus.cc:1275] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.042642  2330 consensus_queue.cc:1048] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.053241  1495 raft_consensus.cc:1275] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.056044  2354 consensus_queue.cc:1048] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.139554  1424 raft_consensus.cc:1275] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.140973  2354 consensus_queue.cc:1048] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.161370  1493 raft_consensus.cc:1275] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.164449  2330 consensus_queue.cc:1048] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.173938  1494 raft_consensus.cc:1275] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.176098  2391 consensus_queue.cc:1048] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:15.189069  1417 raft_consensus.cc:1275] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.191004  2326 consensus_queue.cc:1048] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.230129  1494 raft_consensus.cc:1275] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.232627  2340 consensus_queue.cc:1048] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.249758  1569 raft_consensus.cc:1275] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.252645  2340 consensus_queue.cc:1048] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:15.281055 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.326537  1494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:15.327260  1494 raft_consensus.cc:606] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:15.331990  1494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19086f8d6586412f85d639c45f357757"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:15.332618  1494 raft_consensus.cc:606] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 95782952bdee421094cada4ea955015d
I20260709 12:22:15.349869  1424 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b5431f408774554975e15a5998e2062"
dest_uuid: "95782952bdee421094cada4ea955015d"
mode: GRACEFUL
new_leader_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:58832
I20260709 12:22:15.350514  1424 raft_consensus.cc:606] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Received request to transfer leadership to TS 76217635dae3410b96baf380d386cbee
I20260709 12:22:15.353579  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:15.354224  1568 raft_consensus.cc:606] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:15.354465  2379 raft_consensus.cc:993] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:15.355147  2400 raft_consensus.cc:1081] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:15.356773  1424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19086f8d6586412f85d639c45f357757"
dest_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:57318
I20260709 12:22:15.357358  1424 raft_consensus.cc:493] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.357446  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:15.357631  1424 raft_consensus.cc:3060] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.358106  1568 raft_consensus.cc:606] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:15.359308 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:15.360778 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:15.361475 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.363289  1424 raft_consensus.cc:515] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.365463  1424 leader_election.cc:290] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:15.365852  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19086f8d6586412f85d639c45f357757" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:15.366601  1493 raft_consensus.cc:3055] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.366928  1493 raft_consensus.cc:740] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:15.366856  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19086f8d6586412f85d639c45f357757" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:15.368335  1568 raft_consensus.cc:3060] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.367847  1493 consensus_queue.cc:260] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.370463  1493 raft_consensus.cc:3060] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.378964  1493 raft_consensus.cc:2468] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:15.380360  1388 leader_election.cc:304] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:15.381323  2340 raft_consensus.cc:2804] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.381620  1568 raft_consensus.cc:2468] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:15.381846  2340 raft_consensus.cc:697] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:15.383037  2340 consensus_queue.cc:237] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.383931  2379 raft_consensus.cc:993] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:15.384505  2379 raft_consensus.cc:1081] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:15.386013  2192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42794
I20260709 12:22:15.386569  2192 raft_consensus.cc:493] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.386948  2192 raft_consensus.cc:3060] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.394325  2326 raft_consensus.cc:993] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:15.394881  2326 raft_consensus.cc:1081] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:15.394665  2192 raft_consensus.cc:515] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:15.396759  2189 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42804
I20260709 12:22:15.396941  2192 leader_election.cc:290] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:15.397382  2189 raft_consensus.cc:493] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.397789  2189 raft_consensus.cc:3060] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.398322  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:15.398550  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f190b64b2c4391b5e05cf0854cce5b" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:15.398981  1494 raft_consensus.cc:3055] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.399137  1424 raft_consensus.cc:3060] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.399358  1494 raft_consensus.cc:740] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:15.400441  1494 consensus_queue.cc:260] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:15.402240  1494 raft_consensus.cc:3060] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.409432  2189 raft_consensus.cc:515] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.407541  1424 raft_consensus.cc:2468] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:15.415180  2143 leader_election.cc:304] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:15.415396  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:15.416038  1568 raft_consensus.cc:3055] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.416433  1568 raft_consensus.cc:740] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:15.416703  2330 raft_consensus.cc:2804] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.420147  2330 raft_consensus.cc:697] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:15.421121  2330 consensus_queue.cc:237] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:15.421746  2189 leader_election.cc:290] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:15.426512  1568 consensus_queue.cc:260] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.427928  1568 raft_consensus.cc:3060] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.434003  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe4ebb71004bea9cd778d734ff5dff" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:15.434783  1493 raft_consensus.cc:3060] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.440285  1243 catalog_manager.cc:5697] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 2 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.444128  2398 raft_consensus.cc:993] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d: : Instructing follower 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:15.444810  2398 raft_consensus.cc:1081] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Signalling peer 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:15.446511  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b5431f408774554975e15a5998e2062"
dest_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:45776
I20260709 12:22:15.447103  1569 raft_consensus.cc:493] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.447470  1569 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.450754  1494 raft_consensus.cc:2468] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:15.457336  1493 raft_consensus.cc:2468] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:15.452831  1568 raft_consensus.cc:2468] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:15.464354  1569 raft_consensus.cc:515] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.464983  2143 leader_election.cc:304] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:15.466967  2393 raft_consensus.cc:2804] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.467455  2393 raft_consensus.cc:697] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:15.468480  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:15.469612  1424 raft_consensus.cc:3055] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:15.470125  1424 raft_consensus.cc:740] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:15.470928  1424 consensus_queue.cc:260] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.468613  2393 consensus_queue.cc:237] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.472009  1424 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.490160  1241 catalog_manager.cc:5697] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.505225  1569 leader_election.cc:290] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:15.507925  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b5431f408774554975e15a5998e2062" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:15.508297  1424 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 3.
I20260709 12:22:15.509555  2326 raft_consensus.cc:993] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:15.516117  1493 raft_consensus.cc:3060] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.516172  2321 raft_consensus.cc:1081] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:15.516619  1537 leader_election.cc:304] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:15.517547  2321 raft_consensus.cc:2804] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:15.518121  2321 raft_consensus.cc:697] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 3 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:15.519055  2321 consensus_queue.cc:237] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:15.519834  2192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42804
I20260709 12:22:15.520426  2192 raft_consensus.cc:493] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.520789  2192 raft_consensus.cc:3060] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.531693  2192 raft_consensus.cc:515] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:15.534976  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:15.535176  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9ff4e1da1794b068ed21d8067b023b0" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:15.536206  1494 raft_consensus.cc:3060] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.533485  2192 leader_election.cc:290] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:15.540937  1493 raft_consensus.cc:2468] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 3.
I20260709 12:22:15.545518  1568 raft_consensus.cc:3055] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.545948  1568 raft_consensus.cc:740] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:15.548166  1494 raft_consensus.cc:2468] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:15.550081  2143 leader_election.cc:304] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:15.551471  2393 raft_consensus.cc:2804] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.552656  2393 raft_consensus.cc:697] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:15.547075  1568 consensus_queue.cc:260] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:15.554136  2393 consensus_queue.cc:237] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:15.561527  1568 raft_consensus.cc:3060] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.583199  1241 catalog_manager.cc:5697] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 2 to 3, leader changed from 95782952bdee421094cada4ea955015d (127.23.217.66) to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 3 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.593362  1568 raft_consensus.cc:2468] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:15.596469  1243 catalog_manager.cc:5697] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.602178  1243 catalog_manager.cc:5697] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.773788  1568 raft_consensus.cc:1275] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.775352  2339 consensus_queue.cc:1048] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.801198  1493 raft_consensus.cc:1275] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.803304  2340 consensus_queue.cc:1048] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.873432  1424 raft_consensus.cc:1275] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.875337  2393 consensus_queue.cc:1048] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.887127  1493 raft_consensus.cc:1275] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.890754  2354 consensus_queue.cc:1048] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:15.950163  1493 raft_consensus.cc:1275] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:15.951535  2326 consensus_queue.cc:1048] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:15.961839  1424 raft_consensus.cc:1275] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:15.973660  2391 consensus_queue.cc:1048] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:16.013264  1494 raft_consensus.cc:1275] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.014700  2393 consensus_queue.cc:1048] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:16.018126  1493 raft_consensus.cc:1275] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.019619  2330 consensus_queue.cc:1048] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:16.037002  1569 raft_consensus.cc:1275] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.038807  2424 consensus_queue.cc:1048] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:16.043778  1568 raft_consensus.cc:1275] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:16.045608  2424 consensus_queue.cc:1048] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:16.448443 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:16.490079  1493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:16.490697  1493 raft_consensus.cc:606] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 76217635dae3410b96baf380d386cbee
I20260709 12:22:16.494645  1417 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33"
dest_uuid: "95782952bdee421094cada4ea955015d"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:58832
I20260709 12:22:16.495303  1417 raft_consensus.cc:606] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:16.498862  1493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:16.499585  1493 raft_consensus.cc:606] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 76217635dae3410b96baf380d386cbee
I20260709 12:22:16.503134  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:16.503715  1569 raft_consensus.cc:606] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 95782952bdee421094cada4ea955015d
I20260709 12:22:16.504998 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:22:16.506744 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:16.507491 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:16.584792  2398 raft_consensus.cc:993] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:16.585366  2398 raft_consensus.cc:1081] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:16.587050  2189 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42790
I20260709 12:22:16.587561  2189 raft_consensus.cc:493] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:16.587872  2189 raft_consensus.cc:3060] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.592731  2189 raft_consensus.cc:515] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:16.594285  2189 leader_election.cc:290] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:16.595285  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:16.595816  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b11fecd06b84bfba7a373d360972a33" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:16.596128  1569 raft_consensus.cc:3060] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.596597  1417 raft_consensus.cc:3055] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:16.597024  1417 raft_consensus.cc:740] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:16.597937  1417 consensus_queue.cc:260] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:16.599227  1417 raft_consensus.cc:3060] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.610935  1417 raft_consensus.cc:2468] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:16.613345  1569 raft_consensus.cc:2468] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:16.613555  2143 leader_election.cc:304] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:16.618453  2424 raft_consensus.cc:2804] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:16.618985  2424 raft_consensus.cc:697] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:16.619956  2424 consensus_queue.cc:237] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:16.632306  1241 catalog_manager.cc:5697] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 95782952bdee421094cada4ea955015d (127.23.217.66) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:16.890652  2391 raft_consensus.cc:993] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee: : Instructing follower 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:16.891238  2321 raft_consensus.cc:1081] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 95782952bdee421094cada4ea955015d to start an election
I20260709 12:22:16.892894  1417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032"
dest_uuid: "95782952bdee421094cada4ea955015d"
 from {username='slave'} at 127.0.0.1:57328
I20260709 12:22:16.893576  1417 raft_consensus.cc:493] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:16.893893  1417 raft_consensus.cc:3060] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.899482  1417 raft_consensus.cc:515] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:16.901129  1417 leader_election.cc:290] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Requested vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:16.901983  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:16.902256  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d4cf02d3404f36b94ffb724837b032" candidate_uuid: "95782952bdee421094cada4ea955015d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:16.902799  1494 raft_consensus.cc:3060] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.903155  1568 raft_consensus.cc:3055] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:16.903757  1568 raft_consensus.cc:740] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:16.904825  1568 consensus_queue.cc:260] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:16.906193  1568 raft_consensus.cc:3060] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.907955  2400 raft_consensus.cc:993] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:16.908524  2400 raft_consensus.cc:1081] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:16.912322  1494 raft_consensus.cc:2468] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:16.913558  1388 leader_election.cc:304] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:16.914471  2416 raft_consensus.cc:2804] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:16.915040  2416 raft_consensus.cc:697] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Becoming Leader. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:16.915985  2416 consensus_queue.cc:237] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:16.917214  1568 raft_consensus.cc:2468] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95782952bdee421094cada4ea955015d in term 2.
I20260709 12:22:16.918695  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3"
dest_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:45772
I20260709 12:22:16.919224  1569 raft_consensus.cc:493] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:16.919554  1569 raft_consensus.cc:3060] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.926219  1569 raft_consensus.cc:515] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:16.929141  1569 leader_election.cc:290] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:16.930351  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:16.931022  1417 raft_consensus.cc:3060] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.931295  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ebdd33fb3c0420e8115507583bd63e3" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:16.931864  1493 raft_consensus.cc:3055] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:16.932156  1493 raft_consensus.cc:740] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:16.932873  1493 consensus_queue.cc:260] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:16.932880  1241 catalog_manager.cc:5697] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 95782952bdee421094cada4ea955015d (127.23.217.66). New cstate: current_term: 2 leader_uuid: "95782952bdee421094cada4ea955015d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:16.934201  1493 raft_consensus.cc:3060] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:16.937603  1417 raft_consensus.cc:2468] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:16.938885  1537 leader_election.cc:304] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:16.939692  2321 raft_consensus.cc:2804] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:16.940238  2321 raft_consensus.cc:697] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:16.941032  2321 consensus_queue.cc:237] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:16.943090  1493 raft_consensus.cc:2468] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:16.952037  1241 catalog_manager.cc:5697] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 2 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:16.997603  2367 raft_consensus.cc:993] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:16.998152  2400 raft_consensus.cc:1081] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 76217635dae3410b96baf380d386cbee to start an election
I20260709 12:22:16.999703  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70"
dest_uuid: "76217635dae3410b96baf380d386cbee"
 from {username='slave'} at 127.0.0.1:45772
I20260709 12:22:17.000214  1569 raft_consensus.cc:493] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.000576  1569 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.008462  1569 raft_consensus.cc:515] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:17.010244  1569 leader_election.cc:290] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:17.011379  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:17.012053  1417 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.012164  1494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "76217635dae3410b96baf380d386cbee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:17.012784  1494 raft_consensus.cc:3055] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.013093  1494 raft_consensus.cc:740] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:17.013867  1494 consensus_queue.cc:260] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:17.015200  1494 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.020088  1417 raft_consensus.cc:2468] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:17.021250  1537 leader_election.cc:304] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:17.022053  2321 raft_consensus.cc:2804] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.022856  1494 raft_consensus.cc:2468] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76217635dae3410b96baf380d386cbee in term 2.
I20260709 12:22:17.032471  2321 raft_consensus.cc:697] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Becoming Leader. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:17.033514  2321 consensus_queue.cc:237] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:17.043495  1241 catalog_manager.cc:5697] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 76217635dae3410b96baf380d386cbee (127.23.217.68). New cstate: current_term: 2 leader_uuid: "76217635dae3410b96baf380d386cbee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.084807  1424 raft_consensus.cc:1275] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.086419  2393 consensus_queue.cc:1048] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:17.101825  1568 raft_consensus.cc:1275] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.110586  2424 consensus_queue.cc:1048] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:22:17.335047  1494 raft_consensus.cc:1275] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.336596  2436 consensus_queue.cc:1048] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:17.356415  1424 raft_consensus.cc:1275] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.358258  2447 consensus_queue.cc:1048] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:17.383843  1568 raft_consensus.cc:1275] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.385202  2416 consensus_queue.cc:1048] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:17.397086  1493 raft_consensus.cc:1275] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 95782952bdee421094cada4ea955015d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.399354  2339 consensus_queue.cc:1048] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:17.465540  1424 raft_consensus.cc:1275] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.467085  2321 consensus_queue.cc:1048] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:17.477320  1494 raft_consensus.cc:1275] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 76217635dae3410b96baf380d386cbee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.479058  2447 consensus_queue.cc:1048] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:17.558738 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:17.605537  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "e8246e0558124e98a58932ad7f4676e3"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:17.606235  1569 raft_consensus.cc:606] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Received request to transfer leadership to TS e8246e0558124e98a58932ad7f4676e3
I20260709 12:22:17.611541  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a"
dest_uuid: "76217635dae3410b96baf380d386cbee"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:36958
I20260709 12:22:17.612177  1568 raft_consensus.cc:606] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:17.614791  1424 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295"
dest_uuid: "95782952bdee421094cada4ea955015d"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:58832
I20260709 12:22:17.615267  1424 raft_consensus.cc:606] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:17.616408 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:22:17.618345 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:17.619105 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:17.950444  2436 raft_consensus.cc:993] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee: : Instructing follower e8246e0558124e98a58932ad7f4676e3 to start an election
I20260709 12:22:17.951066  2447 raft_consensus.cc:1081] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Signalling peer e8246e0558124e98a58932ad7f4676e3 to start an election
I20260709 12:22:17.953410  1493 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
 from {username='slave'} at 127.0.0.1:47092
I20260709 12:22:17.954003  1493 raft_consensus.cc:493] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.954342  1493 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.958525  2436 raft_consensus.cc:993] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:17.959142  2447 raft_consensus.cc:1081] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:17.960747  2193 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42804
I20260709 12:22:17.961365  2193 raft_consensus.cc:493] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.961189  1493 raft_consensus.cc:515] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:17.961740  2193 raft_consensus.cc:3060] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.963402  1493 leader_election.cc:290] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 3 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), 76217635dae3410b96baf380d386cbee (127.23.217.68:37405)
I20260709 12:22:17.964311  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:17.964561  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d20d1ea2914a929e506bd8dd2d5e70" candidate_uuid: "e8246e0558124e98a58932ad7f4676e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:17.965003  1417 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.965299  1568 raft_consensus.cc:3055] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:17.965675  1568 raft_consensus.cc:740] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:17.966578  1568 consensus_queue.cc:260] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:17.967955  1568 raft_consensus.cc:3060] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.970198  2193 raft_consensus.cc:515] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:17.970822  1417 raft_consensus.cc:2468] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 3.
I20260709 12:22:17.975311  1463 leader_election.cc:304] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95782952bdee421094cada4ea955015d, e8246e0558124e98a58932ad7f4676e3; no voters: 
I20260709 12:22:17.975590  1568 raft_consensus.cc:2468] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8246e0558124e98a58932ad7f4676e3 in term 3.
I20260709 12:22:17.975735  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:17.976354  1569 raft_consensus.cc:3055] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.976670  1569 raft_consensus.cc:740] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76217635dae3410b96baf380d386cbee, State: Running, Role: LEADER
I20260709 12:22:17.978338  2367 raft_consensus.cc:2804] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:17.978937  2367 raft_consensus.cc:697] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 3 LEADER]: Becoming Leader. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:17.979805  2367 consensus_queue.cc:237] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } }
I20260709 12:22:17.977507  1569 consensus_queue.cc:260] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:17.984167  1569 raft_consensus.cc:3060] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.992518  1569 raft_consensus.cc:2468] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:17.993637  2141 leader_election.cc:304] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:22:17.994573  2193 leader_election.cc:290] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:17.994158  1243 catalog_manager.cc:5697] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 reported cstate change: term changed from 2 to 3, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to e8246e0558124e98a58932ad7f4676e3 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "e8246e0558124e98a58932ad7f4676e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.994562  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eef9c46f2ac4be7a66d7f63d7beb50a" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:17.995630  1424 raft_consensus.cc:3060] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.996002  2443 raft_consensus.cc:2804] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.996536  2443 raft_consensus.cc:697] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:17.997455  2443 consensus_queue.cc:237] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:18.003826  1424 raft_consensus.cc:2468] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:18.031448  1243 catalog_manager.cc:5697] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 76217635dae3410b96baf380d386cbee (127.23.217.68) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.220642  2339 raft_consensus.cc:993] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:18.221231  2455 raft_consensus.cc:1081] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:18.223150  2193 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42790
I20260709 12:22:18.223747  2193 raft_consensus.cc:493] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.224177  2193 raft_consensus.cc:3060] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.229988  2193 raft_consensus.cc:515] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:18.232686  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76217635dae3410b96baf380d386cbee"
I20260709 12:22:18.232946  1417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f1594b2adc4d949964379d8b1e5295" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:18.233296  1569 raft_consensus.cc:3060] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.233772  1417 raft_consensus.cc:3055] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.234143  1417 raft_consensus.cc:740] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95782952bdee421094cada4ea955015d, State: Running, Role: LEADER
I20260709 12:22:18.235046  1417 consensus_queue.cc:260] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:18.236263  1417 raft_consensus.cc:3060] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.239274  1569 raft_consensus.cc:2468] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:18.232301  2193 leader_election.cc:290] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 76217635dae3410b96baf380d386cbee (127.23.217.68:37405), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:18.240613  2141 leader_election.cc:304] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76217635dae3410b96baf380d386cbee, 83f9c1c4921946298df299815121637c; no voters: 
I20260709 12:22:18.241503  2443 raft_consensus.cc:2804] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.242040  2443 raft_consensus.cc:697] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:18.242825  1417 raft_consensus.cc:2468] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:18.243040  2443 consensus_queue.cc:237] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:18.254019  1243 catalog_manager.cc:5697] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from 95782952bdee421094cada4ea955015d (127.23.217.66) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.415495  1569 raft_consensus.cc:1275] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.415664  1417 raft_consensus.cc:1275] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.416930  2393 consensus_queue.cc:1048] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.417966  2443 consensus_queue.cc:1048] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.481351  1569 raft_consensus.cc:1275] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Refusing update from remote peer e8246e0558124e98a58932ad7f4676e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:18.482676  2367 consensus_queue.cc:1048] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:18.493546  1424 raft_consensus.cc:1275] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Refusing update from remote peer e8246e0558124e98a58932ad7f4676e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:18.495005  2367 consensus_queue.cc:1048] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:18.644622  1567 raft_consensus.cc:1275] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.646028  2443 consensus_queue.cc:1048] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.659497  1424 raft_consensus.cc:1275] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.660101 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:18.661520  2443 consensus_queue.cc:1048] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:18.721421  1493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb"
dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
mode: GRACEFUL
new_leader_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:43388
I20260709 12:22:18.722080  1493 raft_consensus.cc:606] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Received request to transfer leadership to TS 83f9c1c4921946298df299815121637c
I20260709 12:22:18.723315 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:22:18.724927 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:18.725615 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:18.824196  2453 raft_consensus.cc:993] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3: : Instructing follower 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:18.824879  2453 raft_consensus.cc:1081] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Signalling peer 83f9c1c4921946298df299815121637c to start an election
I20260709 12:22:18.827723  2193 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb"
dest_uuid: "83f9c1c4921946298df299815121637c"
 from {username='slave'} at 127.0.0.1:42794
I20260709 12:22:18.828315  2193 raft_consensus.cc:493] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.828657  2193 raft_consensus.cc:3060] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.833774  2193 raft_consensus.cc:515] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:18.835547  2193 leader_election.cc:290] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Requested vote from peers 95782952bdee421094cada4ea955015d (127.23.217.66:33559), e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099)
I20260709 12:22:18.836768  1493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8246e0558124e98a58932ad7f4676e3"
I20260709 12:22:18.836769  1424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e5f7e35d647437a99929e665c7052eb" candidate_uuid: "83f9c1c4921946298df299815121637c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95782952bdee421094cada4ea955015d"
I20260709 12:22:18.837585  1493 raft_consensus.cc:3055] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.837708  1424 raft_consensus.cc:3060] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.837922  1493 raft_consensus.cc:740] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8246e0558124e98a58932ad7f4676e3, State: Running, Role: LEADER
I20260709 12:22:18.838996  1493 consensus_queue.cc:260] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:18.840282  1493 raft_consensus.cc:3060] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.844457  1424 raft_consensus.cc:2468] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:18.846235  2143 leader_election.cc:304] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d; no voters: 
I20260709 12:22:18.847098  2442 raft_consensus.cc:2804] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.847594  2442 raft_consensus.cc:697] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Becoming Leader. State: Replica: 83f9c1c4921946298df299815121637c, State: Running, Role: LEADER
I20260709 12:22:18.848596  2442 consensus_queue.cc:237] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } }
I20260709 12:22:18.850678  1493 raft_consensus.cc:2468] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f9c1c4921946298df299815121637c in term 2.
I20260709 12:22:18.860834  1243 catalog_manager.cc:5697] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c reported cstate change: term changed from 1 to 2, leader changed from e8246e0558124e98a58932ad7f4676e3 (127.23.217.67) to 83f9c1c4921946298df299815121637c (127.23.217.65). New cstate: current_term: 2 leader_uuid: "83f9c1c4921946298df299815121637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:19.263509  1494 raft_consensus.cc:1275] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:19.265013  2443 consensus_queue.cc:1048] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:19.282976  1424 raft_consensus.cc:1275] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Refusing update from remote peer 83f9c1c4921946298df299815121637c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:19.284130  2443 consensus_queue.cc:1048] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:19.779580 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.825654 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:19.827252 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:19.827996 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.953696 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.994439 24421 tablet_server.cc:179] TabletServer@127.23.217.65:37337 shutting down...
W20260709 12:22:20.010923  1461 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 161 similar messages]
W20260709 12:22:20.015125  1461 consensus_peers.cc:597] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.025852  1461 consensus_peers.cc:597] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.037580  1386 consensus_peers.cc:597] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.049398  1386 consensus_peers.cc:597] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.054145 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:22:20.057757  1461 consensus_peers.cc:597] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.059026 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.060457 24421 raft_consensus.cc:2243] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.061093 24421 raft_consensus.cc:2272] T 012be77ba30c48409e5b4e286fa90432 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.065012 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.065986 24421 raft_consensus.cc:2243] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.066618 24421 raft_consensus.cc:2272] T 09e9e93f05e547be9a7e726a1f4a419f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.070112 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.071587 24421 raft_consensus.cc:2243] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.072926 24421 raft_consensus.cc:2272] T a89da9bea35044babd7f5631e0e7afd8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.076152 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.077271 24421 raft_consensus.cc:2243] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.078053 24421 raft_consensus.cc:2272] T 6beebaa55c4f41b1b2a0bb635654d53a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.081038  1461 consensus_peers.cc:597] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.081667 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.082666 24421 raft_consensus.cc:2243] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:20.083118  1535 consensus_peers.cc:597] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.083276 24421 raft_consensus.cc:2272] T 4e617f860b45428ba84e2c8b7511ae75 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.085831 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.086436 24421 raft_consensus.cc:2243] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.087291 24421 raft_consensus.cc:2272] T d7f1594b2adc4d949964379d8b1e5295 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.090013 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.090834 24421 raft_consensus.cc:2243] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.091496 24421 raft_consensus.cc:2272] T 3da69d29cbf04086b598c879439a9e8d P 83f9c1c4921946298df299815121637c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.094264 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.095614 24421 raft_consensus.cc:2243] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.096087 24421 raft_consensus.cc:2272] T c1bc80953a524e6f844c37036c559c68 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.097954 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.098461 24421 raft_consensus.cc:2243] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.098899 24421 raft_consensus.cc:2272] T 7fa446bfc89d4e82bd94dd6ce5699591 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.101240 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.102051 24421 raft_consensus.cc:2243] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.103087 24421 raft_consensus.cc:2272] T c9ff4e1da1794b068ed21d8067b023b0 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.104549  1386 consensus_peers.cc:597] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.105654 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.106526 24421 raft_consensus.cc:2243] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.107031 24421 raft_consensus.cc:2272] T c396f4bbb0fe496da0c70c8475f9a05e P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.109802 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.110481 24421 raft_consensus.cc:2243] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.110991 24421 raft_consensus.cc:2272] T a6de610052e5443095393415714a09b3 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.114464 24421 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:20.114624  1386 consensus_peers.cc:597] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.115283 24421 raft_consensus.cc:2243] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.116043 24421 raft_consensus.cc:2272] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.118242 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.118891 24421 raft_consensus.cc:2243] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.119474 24421 raft_consensus.cc:2272] T e5a040bff9d840498d2e8f9bf328b13b P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.122625 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.123483 24421 raft_consensus.cc:2243] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.124555 24421 raft_consensus.cc:2272] T e4fe4ebb71004bea9cd778d734ff5dff P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.126339  1386 consensus_peers.cc:597] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.128098 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.129050 24421 raft_consensus.cc:2243] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:20.129225  1461 consensus_peers.cc:597] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.129558 24421 raft_consensus.cc:2272] T 2d2ac19c44af472cbb267513f0ac353d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.133039 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.134862 24421 raft_consensus.cc:2243] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:20.134941  1461 consensus_peers.cc:597] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.135385 24421 raft_consensus.cc:2272] T 984e201fabf540b6a2d1a83a6d186454 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.135486  1535 consensus_peers.cc:597] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.138319 24421 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:20.139003  1386 consensus_peers.cc:597] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.139397 24421 raft_consensus.cc:2243] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.140002 24421 raft_consensus.cc:2272] T 3cda1629f4344ad8b1ebed68a83d0418 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.142186 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.142906 24421 raft_consensus.cc:2243] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.144670 24421 raft_consensus.cc:2272] T 9ec6fa8077604b94a27cbc5c876af6be P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.147157 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.147743 24421 raft_consensus.cc:2243] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.148306 24421 raft_consensus.cc:2272] T fd76a0ea0cf343509a26d9f566ed01a1 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.151224 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.152047 24421 raft_consensus.cc:2243] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.152908 24421 raft_consensus.cc:2272] T 5e1b0961fa034e77877efe85062b2307 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.154934 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.155534 24421 raft_consensus.cc:2243] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.155975 24421 raft_consensus.cc:2272] T 38a329febbdb4a35accb4f17719726dc P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.158885 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.159612 24421 raft_consensus.cc:2243] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.160405 24421 raft_consensus.cc:2272] T d142957b397e4f75b5526996e950d768 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.162376 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.162987 24421 raft_consensus.cc:2243] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.163429 24421 raft_consensus.cc:2272] T 454ec9762c6f480bbc5587abbce5f8f6 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.166725 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.167590 24421 raft_consensus.cc:2243] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.168078 24421 raft_consensus.cc:2272] T c312c9473d0e47e0816684df0c14a27f P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.170562 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.171142 24421 raft_consensus.cc:2243] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.171550 24421 raft_consensus.cc:2272] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.173445 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.174027 24421 raft_consensus.cc:2243] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.174458 24421 raft_consensus.cc:2272] T db3f7b195a584dd1896512e1497e4ead P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.177228 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.177933 24421 raft_consensus.cc:2243] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.178416 24421 raft_consensus.cc:2272] T 61b61c6a29814aad84e50fcb39c26e56 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.179373  1461 consensus_peers.cc:597] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.181443 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.182308 24421 raft_consensus.cc:2243] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.183313 24421 raft_consensus.cc:2272] T 25b2d9d95dad4ef2ae0f868e611b9eab P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.185972 24421 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:20.185904  1535 consensus_peers.cc:597] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.186939 24421 raft_consensus.cc:2243] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.187393 24421 raft_consensus.cc:2272] T fd1ad92910454a39a4ebb1ea41c3c376 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.190307 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.191033 24421 raft_consensus.cc:2243] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.191594 24421 raft_consensus.cc:2272] T 2efe5434a3fc4ba1a2552cc148e23a9a P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.194558 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.195250 24421 raft_consensus.cc:2243] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.195693 24421 raft_consensus.cc:2272] T f68fc83ac5034733b526df7c6ffef4f8 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.198084 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.198711 24421 raft_consensus.cc:2243] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.199112 24421 raft_consensus.cc:2272] T 8576156f1c0b4a5288eb4f9501386641 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.201543 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.202090 24421 raft_consensus.cc:2243] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.203003 24421 raft_consensus.cc:2272] T 00f190b64b2c4391b5e05cf0854cce5b P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.205566 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.206157 24421 raft_consensus.cc:2243] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.207015 24421 raft_consensus.cc:2272] T 0e5f7e35d647437a99929e665c7052eb P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.209661 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.210217 24421 raft_consensus.cc:2243] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.211050 24421 raft_consensus.cc:2272] T 2b11fecd06b84bfba7a373d360972a33 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.212922 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.213439 24421 raft_consensus.cc:2243] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.213752 24421 raft_consensus.cc:2272] T 58e96847ffff43ca9df16b0fb7a718a9 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.215863 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.216529 24421 raft_consensus.cc:2243] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.216969 24421 raft_consensus.cc:2272] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.219069 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.219707 24421 raft_consensus.cc:2243] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.220263 24421 raft_consensus.cc:2272] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.223129 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.223816 24421 raft_consensus.cc:2243] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.224963 24421 raft_consensus.cc:2272] T 0ce2de400513464ca379c256dc7b0e89 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.227360 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.227931 24421 raft_consensus.cc:2243] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.228682 24421 raft_consensus.cc:2272] T 4af17dabf7ac45919d736585909bca81 P 83f9c1c4921946298df299815121637c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.230839 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.231398 24421 raft_consensus.cc:2243] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.231817 24421 raft_consensus.cc:2272] T fac0e5cd0b264bb18307cc2498883adf P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.234259 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.234843 24421 raft_consensus.cc:2243] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.235342 24421 raft_consensus.cc:2272] T 52095deeb4984630a0d2b2ab844fef7d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.237888 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.238463 24421 raft_consensus.cc:2243] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.238958 24421 raft_consensus.cc:2272] T 086651b7e7954168abc2acd51a3ed43d P 83f9c1c4921946298df299815121637c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.240664  1535 consensus_peers.cc:597] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.241334 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:20.241959 24421 raft_consensus.cc:2243] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.242865 24421 raft_consensus.cc:2272] T 54c81e1068f745e09d8d065e567c4b22 P 83f9c1c4921946298df299815121637c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.321208  1461 consensus_peers.cc:597] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.323849  1535 consensus_peers.cc:597] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.327639 24421 tablet_server.cc:196] TabletServer@127.23.217.65:37337 shutdown complete.
W20260709 12:22:20.380259  1535 consensus_peers.cc:597] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.389743  1535 consensus_peers.cc:597] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.406795  1386 consensus_peers.cc:597] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.411770  1535 consensus_peers.cc:597] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.413695 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
W20260709 12:22:20.424491  1461 consensus_peers.cc:597] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.435133  1535 consensus_peers.cc:597] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.462700 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:20.463765 24421 tablet_replica.cc:333] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.464726 24421 raft_consensus.cc:2243] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.465356 24421 raft_consensus.cc:2272] T 4742a96787194d49a2f191cdd0ef8642 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.468371 24421 tablet_replica.cc:333] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d: stopping tablet replica
W20260709 12:22:20.468809  1463 consensus_peers.cc:597] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.468981 24421 raft_consensus.cc:2243] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.469810 24421 raft_consensus.cc:2272] T 2b11fecd06b84bfba7a373d360972a33 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.472446 24421 tablet_replica.cc:333] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.473408 24421 raft_consensus.cc:2243] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.474176 24421 raft_consensus.cc:2272] T 58e96847ffff43ca9df16b0fb7a718a9 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.477542 24421 tablet_replica.cc:333] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d: stopping tablet replica
W20260709 12:22:20.478091  1461 consensus_peers.cc:597] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.478547 24421 raft_consensus.cc:2243] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.479176 24421 raft_consensus.cc:2272] T 7fa446bfc89d4e82bd94dd6ce5699591 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.482477 24421 tablet_replica.cc:333] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.483554 24421 raft_consensus.cc:2243] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.484157 24421 raft_consensus.cc:2272] T 6b5431f408774554975e15a5998e2062 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.487293  1535 consensus_peers.cc:597] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.488230  1537 consensus_peers.cc:597] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.494079 24421 tablet_replica.cc:333] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.495245 24421 raft_consensus.cc:2243] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.495916 24421 raft_consensus.cc:2272] T a89da9bea35044babd7f5631e0e7afd8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.498522  1386 consensus_peers.cc:597] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.502729 24421 tablet_replica.cc:333] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d: stopping tablet replica
W20260709 12:22:20.503767  1463 consensus_peers.cc:597] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.503939 24421 raft_consensus.cc:2243] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.504824 24421 raft_consensus.cc:2272] T fd1ad92910454a39a4ebb1ea41c3c376 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.507630 24421 tablet_replica.cc:333] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.508330 24421 raft_consensus.cc:2243] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.509198 24421 raft_consensus.cc:2272] T 2236e9eab6544f1f9e89679486b7a6cf P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.511333 24421 tablet_replica.cc:333] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.511961 24421 raft_consensus.cc:2243] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.512378 24421 raft_consensus.cc:2272] T d7f1594b2adc4d949964379d8b1e5295 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.514711 24421 tablet_replica.cc:333] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.515352 24421 raft_consensus.cc:2243] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.516518 24421 raft_consensus.cc:2272] T 012be77ba30c48409e5b4e286fa90432 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.520000 24421 tablet_replica.cc:333] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.520889 24421 raft_consensus.cc:2243] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.521518 24421 raft_consensus.cc:2272] T 09e9e93f05e547be9a7e726a1f4a419f P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.525058 24421 tablet_replica.cc:333] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.525995 24421 raft_consensus.cc:2243] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.527204 24421 raft_consensus.cc:2272] T c312c9473d0e47e0816684df0c14a27f P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.529994  1535 consensus_peers.cc:597] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.530494 24421 tablet_replica.cc:333] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.532047 24421 raft_consensus.cc:2243] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.532634 24421 raft_consensus.cc:2272] T 13d20d1ea2914a929e506bd8dd2d5e70 P 95782952bdee421094cada4ea955015d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.535881 24421 tablet_replica.cc:333] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.536707 24421 raft_consensus.cc:2243] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:20.536793  1463 consensus_peers.cc:597] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.537463 24421 raft_consensus.cc:2272] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.538704  1537 consensus_peers.cc:597] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.540733 24421 tablet_replica.cc:333] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.541867 24421 raft_consensus.cc:2243] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.542656 24421 raft_consensus.cc:2272] T 0ce2de400513464ca379c256dc7b0e89 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.547006  1461 consensus_peers.cc:597] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 -> Peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Couldn't send request to peer 83f9c1c4921946298df299815121637c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.550040 24421 tablet_replica.cc:333] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.550984 24421 raft_consensus.cc:2243] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.551515 24421 raft_consensus.cc:2272] T 4af17dabf7ac45919d736585909bca81 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.554529 24421 tablet_replica.cc:333] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d: stopping tablet replica
W20260709 12:22:20.554955  1537 consensus_peers.cc:597] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.555461 24421 raft_consensus.cc:2243] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.556453 24421 raft_consensus.cc:2272] T fac0e5cd0b264bb18307cc2498883adf P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.558758 24421 tablet_replica.cc:333] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.559396 24421 raft_consensus.cc:2243] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.559819 24421 raft_consensus.cc:2272] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.561842 24421 tablet_replica.cc:333] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.562458 24421 raft_consensus.cc:2243] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.563359 24421 raft_consensus.cc:2272] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.565383 24421 tablet_replica.cc:333] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.565966 24421 raft_consensus.cc:2243] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.566381 24421 raft_consensus.cc:2272] T 6beebaa55c4f41b1b2a0bb635654d53a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.568444 24421 tablet_replica.cc:333] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.569233 24421 raft_consensus.cc:2243] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.569759 24421 raft_consensus.cc:2272] T 984e201fabf540b6a2d1a83a6d186454 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.572397 24421 tablet_replica.cc:333] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.573316 24421 raft_consensus.cc:2243] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:20.573377  1463 consensus_peers.cc:597] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.574059 24421 raft_consensus.cc:2272] T 2d2ac19c44af472cbb267513f0ac353d P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.577032 24421 tablet_replica.cc:333] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.577651 24421 raft_consensus.cc:2243] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.578223 24421 raft_consensus.cc:2272] T 9ec6fa8077604b94a27cbc5c876af6be P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.580400 24421 tablet_replica.cc:333] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.581000 24421 raft_consensus.cc:2243] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.581882 24421 raft_consensus.cc:2272] T c396f4bbb0fe496da0c70c8475f9a05e P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.583930 24421 tablet_replica.cc:333] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.584510 24421 raft_consensus.cc:2243] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.585294 24421 raft_consensus.cc:2272] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.587260 24421 tablet_replica.cc:333] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.587863 24421 raft_consensus.cc:2243] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.588410 24421 raft_consensus.cc:2272] T f68fc83ac5034733b526df7c6ffef4f8 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.593518 24421 tablet_replica.cc:333] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.594362 24421 raft_consensus.cc:2243] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:22:20.599388  1537 consensus_peers.cc:597] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.599946 24421 raft_consensus.cc:2272] T 2efe5434a3fc4ba1a2552cc148e23a9a P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.602917 24421 tablet_replica.cc:333] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.603775 24421 raft_consensus.cc:2243] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:22:20.604470  1537 consensus_peers.cc:597] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.605032 24421 raft_consensus.cc:2272] T a85193c93a364d41b3124bde25907dee P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.607692 24421 tablet_replica.cc:333] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.608398 24421 raft_consensus.cc:2243] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.608852 24421 raft_consensus.cc:2272] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.611251 24421 tablet_replica.cc:333] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.612311 24421 raft_consensus.cc:2243] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.613440 24421 raft_consensus.cc:2272] T a6de610052e5443095393415714a09b3 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.616240 24421 tablet_replica.cc:333] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.616881 24421 raft_consensus.cc:2243] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.617314 24421 raft_consensus.cc:2272] T e5a040bff9d840498d2e8f9bf328b13b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.619972 24421 tablet_replica.cc:333] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.620884 24421 raft_consensus.cc:2243] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.621526 24421 raft_consensus.cc:2272] T 8576156f1c0b4a5288eb4f9501386641 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.624562 24421 tablet_replica.cc:333] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.625379 24421 raft_consensus.cc:2243] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:20.625403  1537 consensus_peers.cc:597] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.625950 24421 raft_consensus.cc:2272] T 5e1b0961fa034e77877efe85062b2307 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.628304 24421 tablet_replica.cc:333] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.629105 24421 raft_consensus.cc:2243] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.629680 24421 raft_consensus.cc:2272] T 38a329febbdb4a35accb4f17719726dc P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.632339 24421 tablet_replica.cc:333] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.632903 24421 raft_consensus.cc:2243] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.633836 24421 raft_consensus.cc:2272] T 4e617f860b45428ba84e2c8b7511ae75 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.635919 24421 tablet_replica.cc:333] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.636471 24421 raft_consensus.cc:2243] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.636937 24421 raft_consensus.cc:2272] T 4d9872c5f8bf40259f4247e381e78e10 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.639315 24421 tablet_replica.cc:333] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.639840 24421 raft_consensus.cc:2243] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.640735 24421 raft_consensus.cc:2272] T db3f7b195a584dd1896512e1497e4ead P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.642830 24421 tablet_replica.cc:333] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.643371 24421 raft_consensus.cc:2243] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.644259 24421 raft_consensus.cc:2272] T 59009719f3a54ceebc3c12b1687b9000 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.646349 24421 tablet_replica.cc:333] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.646930 24421 raft_consensus.cc:2243] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.647351 24421 raft_consensus.cc:2272] T 274048e07f194b339d084f93a0259790 P 95782952bdee421094cada4ea955015d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.649413 24421 tablet_replica.cc:333] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.649947 24421 raft_consensus.cc:2243] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.650774 24421 raft_consensus.cc:2272] T c9d4cf02d3404f36b94ffb724837b032 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.652750 24421 tablet_replica.cc:333] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.653312 24421 raft_consensus.cc:2243] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.653882 24421 raft_consensus.cc:2272] T 3ebdd33fb3c0420e8115507583bd63e3 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.656191 24421 tablet_replica.cc:333] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.656879 24421 raft_consensus.cc:2243] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.657397 24421 raft_consensus.cc:2272] T 0e5f7e35d647437a99929e665c7052eb P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.660226 24421 tablet_replica.cc:333] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.660774 24421 raft_consensus.cc:2243] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.661247 24421 raft_consensus.cc:2272] T 00f190b64b2c4391b5e05cf0854cce5b P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.663944 24421 tablet_replica.cc:333] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d: stopping tablet replica
I20260709 12:22:20.664682 24421 raft_consensus.cc:2243] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.665632 24421 raft_consensus.cc:2272] T 19086f8d6586412f85d639c45f357757 P 95782952bdee421094cada4ea955015d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:20.666023  1463 consensus_peers.cc:597] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.689071  1537 consensus_peers.cc:597] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.694803  1463 consensus_peers.cc:597] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.718616  1463 consensus_peers.cc:597] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.734735  1463 consensus_peers.cc:597] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.757025 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
W20260709 12:22:20.797513  1463 consensus_peers.cc:597] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:33559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.820286  1463 consensus_peers.cc:597] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:33559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.845321 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
W20260709 12:22:20.850711  1463 consensus_peers.cc:597] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:33559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.853916  1537 consensus_peers.cc:597] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:33559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.858518  1537 consensus_peers.cc:597] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:20.876611  1537 consensus_peers.cc:597] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.892553 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:20.893625 24421 tablet_replica.cc:333] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.894685 24421 raft_consensus.cc:2243] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.895682 24421 raft_consensus.cc:2272] T 4742a96787194d49a2f191cdd0ef8642 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.898185 24421 tablet_replica.cc:333] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.898945 24421 raft_consensus.cc:2243] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.899482 24421 raft_consensus.cc:2272] T 6b5431f408774554975e15a5998e2062 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.901803 24421 tablet_replica.cc:333] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.902447 24421 raft_consensus.cc:2243] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.902938 24421 raft_consensus.cc:2272] T c9ff4e1da1794b068ed21d8067b023b0 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.905011 24421 tablet_replica.cc:333] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.905669 24421 raft_consensus.cc:2243] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.906625 24421 raft_consensus.cc:2272] T c1bc80953a524e6f844c37036c559c68 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.909013 24421 tablet_replica.cc:333] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.909763 24421 raft_consensus.cc:2243] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.910766 24421 raft_consensus.cc:2272] T a89da9bea35044babd7f5631e0e7afd8 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.913375 24421 tablet_replica.cc:333] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.914005 24421 raft_consensus.cc:2243] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.914598 24421 raft_consensus.cc:2272] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.917308 24421 tablet_replica.cc:333] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
W20260709 12:22:20.917729  1463 consensus_peers.cc:597] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 -> Peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Couldn't send request to peer 95782952bdee421094cada4ea955015d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:33559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.918185 24421 raft_consensus.cc:2243] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.919466 24421 raft_consensus.cc:2272] T 629287ce1cfc44a0b85fcc0dcbde5c07 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.922672 24421 tablet_replica.cc:333] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
W20260709 12:22:20.923470  1537 consensus_peers.cc:597] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:20.923643 24421 raft_consensus.cc:2243] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.924872 24421 raft_consensus.cc:2272] T 09e9e93f05e547be9a7e726a1f4a419f P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.927938 24421 tablet_replica.cc:333] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.929311 24421 raft_consensus.cc:2243] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.929844 24421 raft_consensus.cc:2272] T 086651b7e7954168abc2acd51a3ed43d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.932247 24421 tablet_replica.cc:333] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.932917 24421 raft_consensus.cc:2243] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.933393 24421 raft_consensus.cc:2272] T c312c9473d0e47e0816684df0c14a27f P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.935704 24421 tablet_replica.cc:333] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.936323 24421 raft_consensus.cc:2243] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.937147 24421 raft_consensus.cc:2272] T 13d20d1ea2914a929e506bd8dd2d5e70 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.939289 24421 tablet_replica.cc:333] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.939870 24421 raft_consensus.cc:2243] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.940712 24421 raft_consensus.cc:2272] T 454ec9762c6f480bbc5587abbce5f8f6 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.942948 24421 tablet_replica.cc:333] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.943573 24421 raft_consensus.cc:2243] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.944578 24421 raft_consensus.cc:2272] T 6beebaa55c4f41b1b2a0bb635654d53a P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.947505 24421 tablet_replica.cc:333] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.948177 24421 raft_consensus.cc:2243] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.948745 24421 raft_consensus.cc:2272] T d142957b397e4f75b5526996e950d768 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.950975 24421 tablet_replica.cc:333] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.951772 24421 raft_consensus.cc:2243] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.952330 24421 raft_consensus.cc:2272] T 52095deeb4984630a0d2b2ab844fef7d P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.955111 24421 tablet_replica.cc:333] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.955865 24421 raft_consensus.cc:2243] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.957029 24421 raft_consensus.cc:2272] T 0cd6baed42964ae8a9fbab7c6e8601c0 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.959551 24421 tablet_replica.cc:333] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.960403 24421 raft_consensus.cc:2243] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.960925 24421 raft_consensus.cc:2272] T 2236e9eab6544f1f9e89679486b7a6cf P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.963587 24421 tablet_replica.cc:333] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.964430 24421 raft_consensus.cc:2243] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.964998 24421 raft_consensus.cc:2272] T 3da69d29cbf04086b598c879439a9e8d P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.967614 24421 tablet_replica.cc:333] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.968387 24421 raft_consensus.cc:2243] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.968906 24421 raft_consensus.cc:2272] T 3cda1629f4344ad8b1ebed68a83d0418 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.971647 24421 tablet_replica.cc:333] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.972500 24421 raft_consensus.cc:2243] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:20.973600 24421 raft_consensus.cc:2272] T 984e201fabf540b6a2d1a83a6d186454 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.976601 24421 tablet_replica.cc:333] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.977468 24421 raft_consensus.cc:2243] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.978114 24421 raft_consensus.cc:2272] T fd76a0ea0cf343509a26d9f566ed01a1 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.981369 24421 tablet_replica.cc:333] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.982208 24421 raft_consensus.cc:2243] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.982779 24421 raft_consensus.cc:2272] T 9ec6fa8077604b94a27cbc5c876af6be P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.985569 24421 tablet_replica.cc:333] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.986356 24421 raft_consensus.cc:2243] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.986935 24421 raft_consensus.cc:2272] T c396f4bbb0fe496da0c70c8475f9a05e P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.989905 24421 tablet_replica.cc:333] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.990687 24421 raft_consensus.cc:2243] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.991267 24421 raft_consensus.cc:2272] T 1508cd4f1b544868bbb54eb5a6f4a6ff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.994076 24421 tablet_replica.cc:333] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.994864 24421 raft_consensus.cc:2243] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:20.995417 24421 raft_consensus.cc:2272] T 5e1b0961fa034e77877efe85062b2307 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:20.998214 24421 tablet_replica.cc:333] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:20.998997 24421 raft_consensus.cc:2243] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.000128 24421 raft_consensus.cc:2272] T 38a329febbdb4a35accb4f17719726dc P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.003014 24421 tablet_replica.cc:333] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.003814 24421 raft_consensus.cc:2243] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.004343 24421 raft_consensus.cc:2272] T 4d9872c5f8bf40259f4247e381e78e10 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.007354 24421 tablet_replica.cc:333] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.008194 24421 raft_consensus.cc:2243] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.008728 24421 raft_consensus.cc:2272] T a85193c93a364d41b3124bde25907dee P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.011760 24421 tablet_replica.cc:333] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.012629 24421 raft_consensus.cc:2243] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.013684 24421 raft_consensus.cc:2272] T e5a040bff9d840498d2e8f9bf328b13b P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.016860 24421 tablet_replica.cc:333] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
W20260709 12:22:21.017570  1537 consensus_peers.cc:597] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:21.017653 24421 raft_consensus.cc:2243] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.018400 24421 raft_consensus.cc:2272] T 59009719f3a54ceebc3c12b1687b9000 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.021279 24421 tablet_replica.cc:333] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.022081 24421 raft_consensus.cc:2243] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.023247 24421 raft_consensus.cc:2272] T 274048e07f194b339d084f93a0259790 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.026208 24421 tablet_replica.cc:333] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.027019 24421 raft_consensus.cc:2243] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.027551 24421 raft_consensus.cc:2272] T c9d4cf02d3404f36b94ffb724837b032 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.030126 24421 tablet_replica.cc:333] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.030949 24421 raft_consensus.cc:2243] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.031461 24421 raft_consensus.cc:2272] T 3ebdd33fb3c0420e8115507583bd63e3 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.034312 24421 tablet_replica.cc:333] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
W20260709 12:22:21.035025  1537 consensus_peers.cc:597] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:21.035059 24421 raft_consensus.cc:2243] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:21.035753  1537 consensus_peers.cc:597] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:21.035880 24421 raft_consensus.cc:2272] T 25b2d9d95dad4ef2ae0f868e611b9eab P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.038573 24421 tablet_replica.cc:333] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.039117 24421 raft_consensus.cc:2243] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.039499 24421 raft_consensus.cc:2272] T 61b61c6a29814aad84e50fcb39c26e56 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.041481 24421 tablet_replica.cc:333] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.042052 24421 raft_consensus.cc:2243] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.042418 24421 raft_consensus.cc:2272] T 4af17dabf7ac45919d736585909bca81 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.044482 24421 tablet_replica.cc:333] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.044975 24421 raft_consensus.cc:2243] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.045431 24421 raft_consensus.cc:2272] T 54c81e1068f745e09d8d065e567c4b22 P e8246e0558124e98a58932ad7f4676e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.047592 24421 tablet_replica.cc:333] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.048121 24421 raft_consensus.cc:2243] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.048554 24421 raft_consensus.cc:2272] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.050644 24421 tablet_replica.cc:333] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.051179 24421 raft_consensus.cc:2243] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.051602 24421 raft_consensus.cc:2272] T e4fe4ebb71004bea9cd778d734ff5dff P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.053519 24421 tablet_replica.cc:333] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.054054 24421 raft_consensus.cc:2243] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.054466 24421 raft_consensus.cc:2272] T 0e5f7e35d647437a99929e665c7052eb P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.056540 24421 tablet_replica.cc:333] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.057189 24421 raft_consensus.cc:2243] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.057639 24421 raft_consensus.cc:2272] T 00f190b64b2c4391b5e05cf0854cce5b P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.059561 24421 tablet_replica.cc:333] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.060014 24421 raft_consensus.cc:2243] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.060371 24421 raft_consensus.cc:2272] T 19086f8d6586412f85d639c45f357757 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.062300 24421 tablet_replica.cc:333] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.062989 24421 raft_consensus.cc:2243] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.063841 24421 raft_consensus.cc:2272] T f68fc83ac5034733b526df7c6ffef4f8 P e8246e0558124e98a58932ad7f4676e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.065730 24421 tablet_replica.cc:333] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3: stopping tablet replica
I20260709 12:22:21.066217 24421 raft_consensus.cc:2243] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.067117 24421 raft_consensus.cc:2272] T 8576156f1c0b4a5288eb4f9501386641 P e8246e0558124e98a58932ad7f4676e3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:21.078032  1537 consensus_peers.cc:597] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:21.105932  2491 raft_consensus.cc:493] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:21.106618  2491 raft_consensus.cc:515] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8246e0558124e98a58932ad7f4676e3" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 43099 } } peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } }
I20260709 12:22:21.108851  2491 leader_election.cc:290] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099), 83f9c1c4921946298df299815121637c (127.23.217.65:37337)
W20260709 12:22:21.111945  1537 leader_election.cc:336] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:22:21.114204  1535 leader_election.cc:336] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:21.114605  1535 leader_election.cc:304] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, e8246e0558124e98a58932ad7f4676e3
I20260709 12:22:21.115550  2491 raft_consensus.cc:2749] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:21.158654 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
W20260709 12:22:21.186101  1537 consensus_peers.cc:597] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.67:43099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:21.228964 24421 tablet_server.cc:179] TabletServer@127.23.217.68:0 shutting down...
I20260709 12:22:21.276993 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:21.277980 24421 tablet_replica.cc:333] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.279121 24421 raft_consensus.cc:2243] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.279809 24421 raft_consensus.cc:2272] T 4742a96787194d49a2f191cdd0ef8642 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.282024 24421 tablet_replica.cc:333] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.282804 24421 raft_consensus.cc:2243] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.283780 24421 raft_consensus.cc:2272] T 58e96847ffff43ca9df16b0fb7a718a9 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.286880 24421 tablet_replica.cc:333] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.287709 24421 raft_consensus.cc:2243] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.287859  2491 raft_consensus.cc:493] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f9c1c4921946298df299815121637c)
I20260709 12:22:21.288172 24421 raft_consensus.cc:2272] T 2b11fecd06b84bfba7a373d360972a33 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.288436  2491 raft_consensus.cc:515] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76217635dae3410b96baf380d386cbee" member_type: VOTER last_known_addr { host: "127.23.217.68" port: 37405 } } peers { permanent_uuid: "83f9c1c4921946298df299815121637c" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 37337 } } peers { permanent_uuid: "95782952bdee421094cada4ea955015d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 33559 } }
I20260709 12:22:21.290253  2491 leader_election.cc:290] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f9c1c4921946298df299815121637c (127.23.217.65:37337), 95782952bdee421094cada4ea955015d (127.23.217.66:33559)
I20260709 12:22:21.290848 24421 tablet_replica.cc:333] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.291994 24421 raft_consensus.cc:2243] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.293062 24421 raft_consensus.cc:2272] T 6b5431f408774554975e15a5998e2062 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:21.295077  1537 leader_election.cc:336] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 95782952bdee421094cada4ea955015d (127.23.217.66:33559): Network error: Client connection negotiation failed: client connection to 127.23.217.66:33559: connect: Connection refused (error 111)
W20260709 12:22:21.295673  1535 leader_election.cc:336] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 83f9c1c4921946298df299815121637c (127.23.217.65:37337): Network error: Client connection negotiation failed: client connection to 127.23.217.65:37337: connect: Connection refused (error 111)
I20260709 12:22:21.295890 24421 tablet_replica.cc:333] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.296130  1535 leader_election.cc:304] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76217635dae3410b96baf380d386cbee; no voters: 83f9c1c4921946298df299815121637c, 95782952bdee421094cada4ea955015d
I20260709 12:22:21.296793 24421 raft_consensus.cc:2243] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.296828  2491 raft_consensus.cc:2749] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:21.297284  1537 consensus_peers.cc:597] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.67:43099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:21.297516 24421 raft_consensus.cc:2272] T c9ff4e1da1794b068ed21d8067b023b0 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.299652 24421 tablet_replica.cc:333] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.300266 24421 raft_consensus.cc:2243] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.301182 24421 raft_consensus.cc:2272] T 7fa446bfc89d4e82bd94dd6ce5699591 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.303654 24421 tablet_replica.cc:333] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.304292 24421 raft_consensus.cc:2243] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.304756 24421 raft_consensus.cc:2272] T c1bc80953a524e6f844c37036c559c68 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.306811 24421 tablet_replica.cc:333] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.307387 24421 raft_consensus.cc:2243] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.308218 24421 raft_consensus.cc:2272] T 086651b7e7954168abc2acd51a3ed43d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.310122 24421 tablet_replica.cc:333] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.310767 24421 raft_consensus.cc:2243] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.311172 24421 raft_consensus.cc:2272] T 13d20d1ea2914a929e506bd8dd2d5e70 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.312984 24421 tablet_replica.cc:333] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.313550 24421 raft_consensus.cc:2243] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.313987 24421 raft_consensus.cc:2272] T 454ec9762c6f480bbc5587abbce5f8f6 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.316045 24421 tablet_replica.cc:333] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.316758 24421 raft_consensus.cc:2243] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.317186 24421 raft_consensus.cc:2272] T 012be77ba30c48409e5b4e286fa90432 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.319231 24421 tablet_replica.cc:333] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.319782 24421 raft_consensus.cc:2243] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.320212 24421 raft_consensus.cc:2272] T d142957b397e4f75b5526996e950d768 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.322037 24421 tablet_replica.cc:333] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.322705 24421 raft_consensus.cc:2243] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.323150 24421 raft_consensus.cc:2272] T a85193c93a364d41b3124bde25907dee P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.325162 24421 tablet_replica.cc:333] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.325722 24421 raft_consensus.cc:2243] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.326617 24421 raft_consensus.cc:2272] T 52095deeb4984630a0d2b2ab844fef7d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.328976 24421 tablet_replica.cc:333] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.329664 24421 raft_consensus.cc:2243] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.330082 24421 raft_consensus.cc:2272] T 274048e07f194b339d084f93a0259790 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.332166 24421 tablet_replica.cc:333] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.332787 24421 raft_consensus.cc:2243] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.333245 24421 raft_consensus.cc:2272] T 59009719f3a54ceebc3c12b1687b9000 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.335484 24421 tablet_replica.cc:333] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.336102 24421 raft_consensus.cc:2243] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.336536 24421 raft_consensus.cc:2272] T 2236e9eab6544f1f9e89679486b7a6cf P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.338846 24421 tablet_replica.cc:333] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.339418 24421 raft_consensus.cc:2243] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.339844 24421 raft_consensus.cc:2272] T d7f1594b2adc4d949964379d8b1e5295 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.341907 24421 tablet_replica.cc:333] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.342559 24421 raft_consensus.cc:2243] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.343420 24421 raft_consensus.cc:2272] T 3da69d29cbf04086b598c879439a9e8d P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.345467 24421 tablet_replica.cc:333] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.346066 24421 raft_consensus.cc:2243] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.346554 24421 raft_consensus.cc:2272] T 0ce2de400513464ca379c256dc7b0e89 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.349011 24421 tablet_replica.cc:333] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.349627 24421 raft_consensus.cc:2243] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.350107 24421 raft_consensus.cc:2272] T fac0e5cd0b264bb18307cc2498883adf P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.352617 24421 tablet_replica.cc:333] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.353354 24421 raft_consensus.cc:2243] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.354180 24421 raft_consensus.cc:2272] T 4f57f5ec0d79454aa37f1c7cd9e30a3a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:21.354259  1537 consensus_peers.cc:597] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee -> Peer e8246e0558124e98a58932ad7f4676e3 (127.23.217.67:43099): Couldn't send request to peer e8246e0558124e98a58932ad7f4676e3. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.67:43099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:21.356403 24421 tablet_replica.cc:333] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.357029 24421 raft_consensus.cc:2243] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.357476 24421 raft_consensus.cc:2272] T 629287ce1cfc44a0b85fcc0dcbde5c07 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.359480 24421 tablet_replica.cc:333] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.360051 24421 raft_consensus.cc:2243] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.360478 24421 raft_consensus.cc:2272] T f2db5d455eb64aad9a2ab7d8fbf9eeef P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.362409 24421 tablet_replica.cc:333] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.363060 24421 raft_consensus.cc:2243] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.363536 24421 raft_consensus.cc:2272] T 0cd6baed42964ae8a9fbab7c6e8601c0 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.365634 24421 tablet_replica.cc:333] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.366353 24421 raft_consensus.cc:2243] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.366950 24421 raft_consensus.cc:2272] T 1508cd4f1b544868bbb54eb5a6f4a6ff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.369557 24421 tablet_replica.cc:333] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.370302 24421 raft_consensus.cc:2243] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.371170 24421 raft_consensus.cc:2272] T 4d9872c5f8bf40259f4247e381e78e10 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.373185 24421 tablet_replica.cc:333] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.373833 24421 raft_consensus.cc:2243] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.374315 24421 raft_consensus.cc:2272] T 4e617f860b45428ba84e2c8b7511ae75 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.376394 24421 tablet_replica.cc:333] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.376950 24421 raft_consensus.cc:2243] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.377451 24421 raft_consensus.cc:2272] T 4eef9c46f2ac4be7a66d7f63d7beb50a P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.380081 24421 tablet_replica.cc:333] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.380700 24421 raft_consensus.cc:2243] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.381130 24421 raft_consensus.cc:2272] T a6de610052e5443095393415714a09b3 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.383323 24421 tablet_replica.cc:333] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.383833 24421 raft_consensus.cc:2243] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.384660 24421 raft_consensus.cc:2272] T 3cda1629f4344ad8b1ebed68a83d0418 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.386729 24421 tablet_replica.cc:333] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.387231 24421 raft_consensus.cc:2243] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.388214 24421 raft_consensus.cc:2272] T 2d2ac19c44af472cbb267513f0ac353d P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.390448 24421 tablet_replica.cc:333] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.391108 24421 raft_consensus.cc:2243] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.392007 24421 raft_consensus.cc:2272] T fd76a0ea0cf343509a26d9f566ed01a1 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.394028 24421 tablet_replica.cc:333] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.394637 24421 raft_consensus.cc:2243] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.395044 24421 raft_consensus.cc:2272] T 54c81e1068f745e09d8d065e567c4b22 P 76217635dae3410b96baf380d386cbee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.397172 24421 tablet_replica.cc:333] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.397701 24421 raft_consensus.cc:2243] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.398108 24421 raft_consensus.cc:2272] T 2efe5434a3fc4ba1a2552cc148e23a9a P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.400704 24421 tablet_replica.cc:333] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.401412 24421 raft_consensus.cc:2243] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.402448 24421 raft_consensus.cc:2272] T cb6c9c94c1f54a0ab4c2e3266ce29f1c P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.404933 24421 tablet_replica.cc:333] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.405426 24421 raft_consensus.cc:2243] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.405833 24421 raft_consensus.cc:2272] T db3f7b195a584dd1896512e1497e4ead P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.407920 24421 tablet_replica.cc:333] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.408438 24421 raft_consensus.cc:2243] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.408819 24421 raft_consensus.cc:2272] T 25b2d9d95dad4ef2ae0f868e611b9eab P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.410741 24421 tablet_replica.cc:333] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.411209 24421 raft_consensus.cc:2243] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.411993 24421 raft_consensus.cc:2272] T fd1ad92910454a39a4ebb1ea41c3c376 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.414016 24421 tablet_replica.cc:333] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.414681 24421 raft_consensus.cc:2243] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.415495 24421 raft_consensus.cc:2272] T 61b61c6a29814aad84e50fcb39c26e56 P 76217635dae3410b96baf380d386cbee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.417629 24421 tablet_replica.cc:333] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.418138 24421 raft_consensus.cc:2243] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.418602 24421 raft_consensus.cc:2272] T c9d4cf02d3404f36b94ffb724837b032 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.420873 24421 tablet_replica.cc:333] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.421648 24421 raft_consensus.cc:2243] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.422901 24421 raft_consensus.cc:2272] T 3ebdd33fb3c0420e8115507583bd63e3 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.425279 24421 tablet_replica.cc:333] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.425818 24421 raft_consensus.cc:2243] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.426268 24421 raft_consensus.cc:2272] T e4fe4ebb71004bea9cd778d734ff5dff P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.428413 24421 tablet_replica.cc:333] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee: stopping tablet replica
I20260709 12:22:21.428934 24421 raft_consensus.cc:2243] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:21.429394 24421 raft_consensus.cc:2272] T 19086f8d6586412f85d639c45f357757 P 76217635dae3410b96baf380d386cbee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.503305 24421 tablet_server.cc:196] TabletServer@127.23.217.68:0 shutdown complete.
I20260709 12:22:21.570416 24421 master.cc:562] Master@127.23.217.126:45747 shutting down...
I20260709 12:22:21.607586 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:21.608153 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:21.608479 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6485732348384e9795affe8a32c3126e: stopping tablet replica
I20260709 12:22:21.631469 24421 master.cc:584] Master@127.23.217.126:45747 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (34084 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:22:21.730157 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:42545
I20260709 12:22:21.731343 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:21.738097  2496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:21.738173  2497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:21.739219  2499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:21.740499 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:21.741662 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:21.741853 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:21.742058 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599741742043 us; error 0 us; skew 500 ppm
I20260709 12:22:21.742686 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:21.745229 24421 webserver.cc:533] Webserver started at http://127.23.217.126:45139/ using document root <none> and password file <none>
I20260709 12:22:21.745715 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:21.745898 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:21.746187 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:21.747686 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "83a0e1d1a7d54d5290dba0bb1b180006"
format_stamp: "Formatted at 2026-07-09 12:22:21 on dist-test-slave-lg8f"
I20260709 12:22:21.752192 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:22:21.755656  2504 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:21.756438 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:22:21.756744 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "83a0e1d1a7d54d5290dba0bb1b180006"
format_stamp: "Formatted at 2026-07-09 12:22:21 on dist-test-slave-lg8f"
I20260709 12:22:21.757010 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:21.767974 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:21.769152 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:21.810873 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:42545
I20260709 12:22:21.811051  2555 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:42545 every 8 connection(s)
I20260709 12:22:21.815137  2556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:21.827190  2556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006: Bootstrap starting.
I20260709 12:22:21.832650  2556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:21.837563  2556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006: No bootstrap required, opened a new log
I20260709 12:22:21.840693  2556 raft_consensus.cc:359] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" member_type: VOTER }
I20260709 12:22:21.841248  2556 raft_consensus.cc:385] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:21.841534  2556 raft_consensus.cc:740] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a0e1d1a7d54d5290dba0bb1b180006, State: Initialized, Role: FOLLOWER
I20260709 12:22:21.842279  2556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" member_type: VOTER }
I20260709 12:22:21.842901  2556 raft_consensus.cc:399] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:21.843214  2556 raft_consensus.cc:493] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:21.843538  2556 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:21.850690  2556 raft_consensus.cc:515] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" member_type: VOTER }
I20260709 12:22:21.851377  2556 leader_election.cc:304] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83a0e1d1a7d54d5290dba0bb1b180006; no voters: 
I20260709 12:22:21.853019  2556 leader_election.cc:290] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:21.853539  2559 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:21.855159  2559 raft_consensus.cc:697] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 1 LEADER]: Becoming Leader. State: Replica: 83a0e1d1a7d54d5290dba0bb1b180006, State: Running, Role: LEADER
I20260709 12:22:21.855901  2559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" member_type: VOTER }
I20260709 12:22:21.857074  2556 sys_catalog.cc:565] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:21.860555  2560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" member_type: VOTER } }
I20260709 12:22:21.860646  2561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83a0e1d1a7d54d5290dba0bb1b180006. Latest consensus state: current_term: 1 leader_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a0e1d1a7d54d5290dba0bb1b180006" member_type: VOTER } }
I20260709 12:22:21.861555  2560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:21.862967  2561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:21.868561  2565 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:21.873324  2565 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:21.877408 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:21.882474  2565 catalog_manager.cc:1383] Generated new cluster ID: 26de3b5afdf94c159591a7b2028c633a
I20260709 12:22:21.882877  2565 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:21.939467  2565 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:21.940961  2565 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:21.954200  2565 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006: Generated new TSK 0
I20260709 12:22:21.954958  2565 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:22.008751 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:22.015440  2577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:22.016597  2578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:22.018090  2580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:22.018971 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:22.019923 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:22.020112 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:22.020303 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599742020290 us; error 0 us; skew 500 ppm
I20260709 12:22:22.020917 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:22.023622 24421 webserver.cc:533] Webserver started at http://127.23.217.65:36555/ using document root <none> and password file <none>
I20260709 12:22:22.024118 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:22.024302 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:22.024637 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:22.025671 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "0cf90e155681407ea8ab2905f6bd31b0"
format_stamp: "Formatted at 2026-07-09 12:22:22 on dist-test-slave-lg8f"
I20260709 12:22:22.030198 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:22:22.033627  2585 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.034462 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:22.034936 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "0cf90e155681407ea8ab2905f6bd31b0"
format_stamp: "Formatted at 2026-07-09 12:22:22 on dist-test-slave-lg8f"
I20260709 12:22:22.035220 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:22.055485 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:22.056725 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:22.058332 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:22.060830 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:22.061075 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.061314 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:22.061503 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.104677 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:34889
I20260709 12:22:22.104846  2647 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:34889 every 8 connection(s)
I20260709 12:22:22.109826 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:22.117302  2652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:22.119450  2653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:22.122915 24421 server_base.cc:1061] running on GCE node
W20260709 12:22:22.123185  2655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:22.124488 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:22.124765 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:22.124979 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599742124959 us; error 0 us; skew 500 ppm
I20260709 12:22:22.125710 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:22.127624  2648 heartbeater.cc:344] Connected to a master server at 127.23.217.126:42545
I20260709 12:22:22.128039  2648 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:22.128669 24421 webserver.cc:533] Webserver started at http://127.23.217.66:37773/ using document root <none> and password file <none>
I20260709 12:22:22.129062  2648 heartbeater.cc:507] Master 127.23.217.126:42545 requested a full tablet report, sending...
I20260709 12:22:22.129222 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:22.129592 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:22.129863 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:22.131191 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
format_stamp: "Formatted at 2026-07-09 12:22:22 on dist-test-slave-lg8f"
I20260709 12:22:22.131672  2521 ts_manager.cc:194] Registered new tserver with Master: 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889)
I20260709 12:22:22.133635  2521 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39052
I20260709 12:22:22.136293 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:22:22.139952  2660 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.140731 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:22.141036 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
format_stamp: "Formatted at 2026-07-09 12:22:22 on dist-test-slave-lg8f"
I20260709 12:22:22.141314 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:22.157557 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:22.158879 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:22.160408 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:22.162645 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:22.162858 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.163089 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:22.163256 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.207530 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:41357
I20260709 12:22:22.207770  2722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:41357 every 8 connection(s)
I20260709 12:22:22.212834 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:22.220286  2726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:22.222162  2727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:22.228645  2729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:22.229252 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:22.229537  2723 heartbeater.cc:344] Connected to a master server at 127.23.217.126:42545
I20260709 12:22:22.230031  2723 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:22.230544 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:22.230860 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:22.231084 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599742231060 us; error 0 us; skew 500 ppm
I20260709 12:22:22.231086  2723 heartbeater.cc:507] Master 127.23.217.126:42545 requested a full tablet report, sending...
I20260709 12:22:22.232035 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:22.233767  2521 ts_manager.cc:194] Registered new tserver with Master: 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:22.235167 24421 webserver.cc:533] Webserver started at http://127.23.217.67:41189/ using document root <none> and password file <none>
I20260709 12:22:22.235904 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:22.235839  2521 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39062
I20260709 12:22:22.236380 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:22.236789 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:22.238447 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "e9352e7947904c43a959a977679d8864"
format_stamp: "Formatted at 2026-07-09 12:22:22 on dist-test-slave-lg8f"
I20260709 12:22:22.243438 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:22:22.246979  2734 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.247889 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 12:22:22.248232 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "e9352e7947904c43a959a977679d8864"
format_stamp: "Formatted at 2026-07-09 12:22:22 on dist-test-slave-lg8f"
I20260709 12:22:22.248538 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:22.269207 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:22.270712 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:22.272128 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:22.274765 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:22.274956 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.275262 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:22.275424 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:22.320411 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:34351
I20260709 12:22:22.320525  2796 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:34351 every 8 connection(s)
I20260709 12:22:22.335525  2797 heartbeater.cc:344] Connected to a master server at 127.23.217.126:42545
I20260709 12:22:22.336004  2797 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:22.336932  2797 heartbeater.cc:507] Master 127.23.217.126:42545 requested a full tablet report, sending...
I20260709 12:22:22.339263  2521 ts_manager.cc:194] Registered new tserver with Master: e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:22.340199 24421 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016347341s
I20260709 12:22:22.340806  2521 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39076
I20260709 12:22:22.367782  2521 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39084:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:22.371125  2521 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:22.444883  2688 tablet_service.cc:1511] Processing CreateTablet for tablet 57ec28a1fdb64950a95df22fc2b70a98 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:22.446756  2688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57ec28a1fdb64950a95df22fc2b70a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.447795  2687 tablet_service.cc:1511] Processing CreateTablet for tablet a737173da1a44edf98cb7483d880e113 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:22.449667  2687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a737173da1a44edf98cb7483d880e113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.451275  2686 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc5e66f25ff411090cd8bb264a2451f (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:22.453220  2686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc5e66f25ff411090cd8bb264a2451f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.457118  2762 tablet_service.cc:1511] Processing CreateTablet for tablet 57ec28a1fdb64950a95df22fc2b70a98 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:22.462805  2762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57ec28a1fdb64950a95df22fc2b70a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.467501  2761 tablet_service.cc:1511] Processing CreateTablet for tablet a737173da1a44edf98cb7483d880e113 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:22.469765  2761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a737173da1a44edf98cb7483d880e113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.469897  2685 tablet_service.cc:1511] Processing CreateTablet for tablet 8f72a8b042f84bf0b1691128bac6b0f4 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:22.471566  2685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f72a8b042f84bf0b1691128bac6b0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.472517  2760 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc5e66f25ff411090cd8bb264a2451f (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:22.474156  2760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc5e66f25ff411090cd8bb264a2451f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.479249  2759 tablet_service.cc:1511] Processing CreateTablet for tablet 8f72a8b042f84bf0b1691128bac6b0f4 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:22.480825  2759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f72a8b042f84bf0b1691128bac6b0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.496196  2684 tablet_service.cc:1511] Processing CreateTablet for tablet 3f0b5e339cec42f699083014deb0f51a (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:22.497901  2684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f0b5e339cec42f699083014deb0f51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.499827  2613 tablet_service.cc:1511] Processing CreateTablet for tablet 57ec28a1fdb64950a95df22fc2b70a98 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:22.501269  2613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57ec28a1fdb64950a95df22fc2b70a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.503531  2612 tablet_service.cc:1511] Processing CreateTablet for tablet a737173da1a44edf98cb7483d880e113 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:22.504886  2612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a737173da1a44edf98cb7483d880e113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.507275  2611 tablet_service.cc:1511] Processing CreateTablet for tablet 6fc5e66f25ff411090cd8bb264a2451f (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:22.508553  2611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fc5e66f25ff411090cd8bb264a2451f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.514778  2609 tablet_service.cc:1511] Processing CreateTablet for tablet 3f0b5e339cec42f699083014deb0f51a (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:22.514981  2761 tablet_service.cc:1511] Processing CreateTablet for tablet 3f0b5e339cec42f699083014deb0f51a (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:22.516129  2761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f0b5e339cec42f699083014deb0f51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.511656  2610 tablet_service.cc:1511] Processing CreateTablet for tablet 8f72a8b042f84bf0b1691128bac6b0f4 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:22.517375  2609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f0b5e339cec42f699083014deb0f51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.520475  2610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f72a8b042f84bf0b1691128bac6b0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.526362  2608 tablet_service.cc:1511] Processing CreateTablet for tablet bdeb55b57c25429f9647fc81d00861da (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:22.528025  2608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdeb55b57c25429f9647fc81d00861da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.530323  2683 tablet_service.cc:1511] Processing CreateTablet for tablet bdeb55b57c25429f9647fc81d00861da (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:22.531795  2683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdeb55b57c25429f9647fc81d00861da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.540769  2758 tablet_service.cc:1511] Processing CreateTablet for tablet bdeb55b57c25429f9647fc81d00861da (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:22.542308  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdeb55b57c25429f9647fc81d00861da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.554380  2812 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:22.569554  2607 tablet_service.cc:1511] Processing CreateTablet for tablet 4511dc9b60d046a49fa1c3a1234cdc29 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:22.571230  2607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4511dc9b60d046a49fa1c3a1234cdc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.577581  2812 tablet_bootstrap.cc:654] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.583323  2813 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:22.599026  2813 tablet_bootstrap.cc:654] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.621467  2684 tablet_service.cc:1511] Processing CreateTablet for tablet 4511dc9b60d046a49fa1c3a1234cdc29 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:22.624089  2684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4511dc9b60d046a49fa1c3a1234cdc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.624439  2814 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:22.629801  2758 tablet_service.cc:1511] Processing CreateTablet for tablet 4511dc9b60d046a49fa1c3a1234cdc29 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:22.631366  2758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4511dc9b60d046a49fa1c3a1234cdc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.632423  2760 tablet_service.cc:1511] Processing CreateTablet for tablet e42c4862903745279ec09f631252173d (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:22.633800  2760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42c4862903745279ec09f631252173d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.634047  2608 tablet_service.cc:1511] Processing CreateTablet for tablet e42c4862903745279ec09f631252173d (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:22.635599  2608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42c4862903745279ec09f631252173d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.635421  2685 tablet_service.cc:1511] Processing CreateTablet for tablet e42c4862903745279ec09f631252173d (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:22.637408  2685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42c4862903745279ec09f631252173d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.641631  2812 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:22.642221  2812 ts_tablet_manager.cc:1403] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.014s
I20260709 12:22:22.644340  2609 tablet_service.cc:1511] Processing CreateTablet for tablet 21eeb3b649044104bf06aa1fcd15cb99 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:22.645749  2609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21eeb3b649044104bf06aa1fcd15cb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.647929  2683 tablet_service.cc:1511] Processing CreateTablet for tablet 21eeb3b649044104bf06aa1fcd15cb99 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:22.648545  2814 tablet_bootstrap.cc:654] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.649408  2683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21eeb3b649044104bf06aa1fcd15cb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.653228  2761 tablet_service.cc:1511] Processing CreateTablet for tablet 21eeb3b649044104bf06aa1fcd15cb99 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:22.654682  2761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21eeb3b649044104bf06aa1fcd15cb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.657617  2812 raft_consensus.cc:359] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.653257  2813 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:22.658437  2812 raft_consensus.cc:385] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.658943  2813 ts_tablet_manager.cc:1403] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.076s	user 0.008s	sys 0.008s
I20260709 12:22:22.663630  2813 raft_consensus.cc:359] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.666057  2610 tablet_service.cc:1511] Processing CreateTablet for tablet 1507d1bea1424ff89b079b67055e8896 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:22.674808  2813 raft_consensus.cc:385] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.675768  2813 raft_consensus.cc:740] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.660012  2812 raft_consensus.cc:740] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.675871  2610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1507d1bea1424ff89b079b67055e8896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.673830  2683 tablet_service.cc:1511] Processing CreateTablet for tablet 1507d1bea1424ff89b079b67055e8896 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:22.679487  2683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1507d1bea1424ff89b079b67055e8896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.680475  2814 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:22.678974  2813 consensus_queue.cc:260] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.681129  2814 ts_tablet_manager.cc:1403] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.000s
I20260709 12:22:22.677557  2812 consensus_queue.cc:260] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.684727  2814 raft_consensus.cc:359] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.685573  2814 raft_consensus.cc:385] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.685448  2759 tablet_service.cc:1511] Processing CreateTablet for tablet 1507d1bea1424ff89b079b67055e8896 (DEFAULT_TABLE table=test-workload [id=451b8a5b0463444893a4ea3e8eef6e22]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:22.686982  2759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1507d1bea1424ff89b079b67055e8896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:22.686079  2814 raft_consensus.cc:740] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.696754  2814 consensus_queue.cc:260] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.704159  2797 heartbeater.cc:499] Master 127.23.217.126:42545 was elected leader, sending a full tablet report...
I20260709 12:22:22.705343  2813 ts_tablet_manager.cc:1434] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.003s
I20260709 12:22:22.706241  2723 heartbeater.cc:499] Master 127.23.217.126:42545 was elected leader, sending a full tablet report...
I20260709 12:22:22.708344  2813 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
W20260709 12:22:22.712481  2724 tablet.cc:2406] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:22.718046  2812 ts_tablet_manager.cc:1434] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.004s
I20260709 12:22:22.719755  2812 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:22.724539  2819 raft_consensus.cc:493] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.725322  2819 raft_consensus.cc:515] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.734874  2812 tablet_bootstrap.cc:654] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.737459  2814 ts_tablet_manager.cc:1434] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.056s	user 0.019s	sys 0.020s
I20260709 12:22:22.738845  2813 tablet_bootstrap.cc:654] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.739125  2648 heartbeater.cc:499] Master 127.23.217.126:42545 was elected leader, sending a full tablet report...
I20260709 12:22:22.743827  2814 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:22.754235  2814 tablet_bootstrap.cc:654] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.759443  2820 raft_consensus.cc:493] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.760255  2820 raft_consensus.cc:515] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.763139  2819 leader_election.cc:290] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9352e7947904c43a959a977679d8864 (127.23.217.67:34351), 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889)
I20260709 12:22:22.768548  2820 leader_election.cc:290] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:22.778475  2813 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:22.779172  2813 ts_tablet_manager.cc:1403] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.071s	user 0.008s	sys 0.005s
I20260709 12:22:22.782315  2813 raft_consensus.cc:359] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.783074  2813 raft_consensus.cc:385] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.783365  2813 raft_consensus.cc:740] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.808794  2813 consensus_queue.cc:260] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.810460  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57ec28a1fdb64950a95df22fc2b70a98" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864" is_pre_election: true
I20260709 12:22:22.811301  2814 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:22.811456  2772 raft_consensus.cc:2468] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 0.
I20260709 12:22:22.811973  2814 ts_tablet_manager.cc:1403] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.068s	user 0.027s	sys 0.032s
I20260709 12:22:22.813328  2663 leader_election.cc:304] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:22.814509  2819 raft_consensus.cc:2804] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:22.814800  2813 ts_tablet_manager.cc:1434] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:22:22.814920  2819 raft_consensus.cc:493] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.815652  2819 raft_consensus.cc:3060] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:22.816184  2813 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:22.815652  2814 raft_consensus.cc:359] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.816679  2814 raft_consensus.cc:385] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.817022  2814 raft_consensus.cc:740] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.817878  2814 consensus_queue.cc:260] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.820303  2814 ts_tablet_manager.cc:1434] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:22.820708  2812 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:22.821285  2812 ts_tablet_manager.cc:1403] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.102s	user 0.010s	sys 0.008s
I20260709 12:22:22.821456  2814 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:22.828611  2819 raft_consensus.cc:515] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.830916  2812 raft_consensus.cc:359] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.831758  2812 raft_consensus.cc:385] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.832055  2812 raft_consensus.cc:740] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.832367  2819 leader_election.cc:290] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 election: Requested vote from peers e9352e7947904c43a959a977679d8864 (127.23.217.67:34351), 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889)
I20260709 12:22:22.829865  2814 tablet_bootstrap.cc:654] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.832849  2812 consensus_queue.cc:260] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.834342  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57ec28a1fdb64950a95df22fc2b70a98" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:22.835120  2772 raft_consensus.cc:3060] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:22.838358  2813 tablet_bootstrap.cc:654] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.844295  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0b5e339cec42f699083014deb0f51a" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
W20260709 12:22:22.846272  2589 leader_election.cc:343] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:22.846968  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0b5e339cec42f699083014deb0f51a" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864" is_pre_election: true
W20260709 12:22:22.848526  2588 leader_election.cc:343] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9352e7947904c43a959a977679d8864 (127.23.217.67:34351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:22.848987  2588 leader_election.cc:304] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0; no voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864
I20260709 12:22:22.850138  2820 raft_consensus.cc:2749] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:22.850509  2812 ts_tablet_manager.cc:1434] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:22:22.852837  2812 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:22.858004  2772 raft_consensus.cc:2468] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 1.
I20260709 12:22:22.861732  2812 tablet_bootstrap.cc:654] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.863433  2663 leader_election.cc:304] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:22.864955  2813 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:22.865010  2819 raft_consensus.cc:2804] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:22.865938  2813 ts_tablet_manager.cc:1403] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 12:22:22.866174  2819 raft_consensus.cc:697] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:22.867671  2819 consensus_queue.cc:237] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:22.873160  2812 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:22.873986  2812 ts_tablet_manager.cc:1403] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.011s
I20260709 12:22:22.877570  2812 raft_consensus.cc:359] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.878319  2812 raft_consensus.cc:385] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.878682  2812 raft_consensus.cc:740] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
W20260709 12:22:22.879499  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:22.879733  2812 consensus_queue.cc:260] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.892760  2821 raft_consensus.cc:493] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.893841  2821 raft_consensus.cc:515] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.895022  2812 ts_tablet_manager.cc:1434] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.004s
I20260709 12:22:22.896461  2812 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:22.901779  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc5e66f25ff411090cd8bb264a2451f" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864" is_pre_election: true
I20260709 12:22:22.904688  2812 tablet_bootstrap.cc:654] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.910053  2813 raft_consensus.cc:359] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.911202  2813 raft_consensus.cc:385] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.911592  2813 raft_consensus.cc:740] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.912611  2813 consensus_queue.cc:260] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.919736  2812 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:22.920444  2812 ts_tablet_manager.cc:1403] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:22:22.922658  2813 ts_tablet_manager.cc:1434] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.056s	user 0.015s	sys 0.003s
I20260709 12:22:22.923669  2813 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:22.924782  2812 raft_consensus.cc:359] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.926250  2812 raft_consensus.cc:385] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.926577  2812 raft_consensus.cc:740] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.927597  2812 consensus_queue.cc:260] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.930117  2812 ts_tablet_manager.cc:1434] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:22:22.930871  2813 tablet_bootstrap.cc:654] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.931170  2812 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:22.943199  2812 tablet_bootstrap.cc:654] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.943840  2813 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:22.944422  2813 ts_tablet_manager.cc:1403] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.011s
I20260709 12:22:22.949270  2818 raft_consensus.cc:493] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.950161  2818 raft_consensus.cc:515] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.953981  2818 leader_election.cc:290] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:22.962540  2821 leader_election.cc:290] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:22.962447  2813 raft_consensus.cc:359] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.964365  2813 raft_consensus.cc:385] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.964684  2813 raft_consensus.cc:740] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.965514  2813 consensus_queue.cc:260] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.973320  2814 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:22.973874  2812 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:22.974089  2814 ts_tablet_manager.cc:1403] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.153s	user 0.062s	sys 0.078s
I20260709 12:22:22.974531  2812 ts_tablet_manager.cc:1403] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.003s
I20260709 12:22:22.973448  2813 ts_tablet_manager.cc:1434] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.002s
I20260709 12:22:22.978010  2813 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:22.979671  2814 raft_consensus.cc:359] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.980530  2814 raft_consensus.cc:385] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.980868  2814 raft_consensus.cc:740] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.981667  2814 consensus_queue.cc:260] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.984037  2814 ts_tablet_manager.cc:1434] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:22:22.985212  2814 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:22.985852  2813 tablet_bootstrap.cc:654] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.977826  2812 raft_consensus.cc:359] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:22.993367  2814 tablet_bootstrap.cc:654] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:22.988994  2812 raft_consensus.cc:385] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:22.997160  2812 raft_consensus.cc:740] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:22.998011  2812 consensus_queue.cc:260] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.000231  2812 ts_tablet_manager.cc:1434] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:22:23.001209  2812 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:23.008872  2623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.009863  2814 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:23.010453  2814 ts_tablet_manager.cc:1403] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
W20260709 12:22:23.011114  2736 leader_election.cc:343] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:23.013995  2814 raft_consensus.cc:359] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.014894  2814 raft_consensus.cc:385] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.017360  2812 tablet_bootstrap.cc:654] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.017824  2813 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:23.015241  2814 raft_consensus.cc:740] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.018445  2813 ts_tablet_manager.cc:1403] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.014s
I20260709 12:22:23.019006  2814 consensus_queue.cc:260] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.022018  2814 ts_tablet_manager.cc:1434] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:22:23.022079  2813 raft_consensus.cc:359] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.023335  2813 raft_consensus.cc:385] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.023535  2814 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:23.023779  2813 raft_consensus.cc:740] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.025137  2813 consensus_queue.cc:260] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.028388  2813 ts_tablet_manager.cc:1434] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:22:23.029757  2813 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:23.032182  2814 tablet_bootstrap.cc:654] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.033054  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.033869  2698 raft_consensus.cc:2468] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.035712  2738 leader_election.cc:304] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 0cf90e155681407ea8ab2905f6bd31b0
I20260709 12:22:23.037497  2818 raft_consensus.cc:2804] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.037990  2818 raft_consensus.cc:493] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.038439  2813 tablet_bootstrap.cc:654] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.041190  2812 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:23.041853  2812 ts_tablet_manager.cc:1403] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.015s
I20260709 12:22:23.045423  2814 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:23.046099  2814 ts_tablet_manager.cc:1403] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.005s
I20260709 12:22:23.046130  2812 raft_consensus.cc:359] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.047464  2812 raft_consensus.cc:385] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.047866  2812 raft_consensus.cc:740] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.048957  2812 consensus_queue.cc:260] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.050269  2814 raft_consensus.cc:359] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.051234  2814 raft_consensus.cc:385] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.051595  2814 raft_consensus.cc:740] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.051859  2813 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:23.052601  2813 ts_tablet_manager.cc:1403] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.009s
I20260709 12:22:23.052707  2814 consensus_queue.cc:260] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.056387  2813 raft_consensus.cc:359] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.057401  2813 raft_consensus.cc:385] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.057814  2813 raft_consensus.cc:740] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.038424  2818 raft_consensus.cc:3060] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.063047  2814 ts_tablet_manager.cc:1434] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 12:22:23.064438  2814 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
W20260709 12:22:23.069142  2663 leader_election.cc:343] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9352e7947904c43a959a977679d8864 (127.23.217.67:34351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:23.072088  2814 tablet_bootstrap.cc:654] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.071448  2813 consensus_queue.cc:260] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.075243  2813 ts_tablet_manager.cc:1434] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.004s
I20260709 12:22:23.076403  2813 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:23.070369  2818 raft_consensus.cc:515] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.085199  2818 leader_election.cc:290] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.086526  2819 raft_consensus.cc:493] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.087215  2819 raft_consensus.cc:515] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.089833  2819 leader_election.cc:290] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:23.092816  2813 tablet_bootstrap.cc:654] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.092702  2812 ts_tablet_manager.cc:1434] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.020s
I20260709 12:22:23.094367  2812 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:23.102596  2812 tablet_bootstrap.cc:654] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.119313  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:23.120415  2698 raft_consensus.cc:3060] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.129632  2698 raft_consensus.cc:2468] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.129846  2623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57ec28a1fdb64950a95df22fc2b70a98" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.130303  2622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57ec28a1fdb64950a95df22fc2b70a98" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.130730  2623 raft_consensus.cc:2468] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 0.
I20260709 12:22:23.131088  2738 leader_election.cc:304] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.131785  2621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc5e66f25ff411090cd8bb264a2451f" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.132157  2846 raft_consensus.cc:2804] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.132512  2621 raft_consensus.cc:2468] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 0.
I20260709 12:22:23.132685  2846 raft_consensus.cc:697] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:23.133584  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.133652  2846 consensus_queue.cc:237] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.136766  2662 leader_election.cc:304] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: e9352e7947904c43a959a977679d8864
W20260709 12:22:23.138016  2662 leader_election.cc:343] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:23.167906  2812 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:23.168658  2812 ts_tablet_manager.cc:1403] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.075s	user 0.017s	sys 0.002s
I20260709 12:22:23.172585  2812 raft_consensus.cc:359] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.173369  2812 raft_consensus.cc:385] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.173662  2812 raft_consensus.cc:740] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.174467  2812 consensus_queue.cc:260] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.177048  2812 ts_tablet_manager.cc:1434] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.007s
I20260709 12:22:23.178088  2812 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:23.183933  2812 tablet_bootstrap.cc:654] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.193403  2812 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:23.193873  2812 ts_tablet_manager.cc:1403] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:22:23.196367  2812 raft_consensus.cc:359] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.197392  2819 raft_consensus.cc:2804] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.199788  2819 raft_consensus.cc:493] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.200137  2819 raft_consensus.cc:3060] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.207134  2819 raft_consensus.cc:515] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.199229  2814 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:23.208323  2814 ts_tablet_manager.cc:1403] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.144s	user 0.023s	sys 0.011s
I20260709 12:22:23.210007  2819 leader_election.cc:290] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:23.210837  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc5e66f25ff411090cd8bb264a2451f" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.211105  2814 raft_consensus.cc:359] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.211802  2814 raft_consensus.cc:385] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.212244  2814 raft_consensus.cc:740] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.212400  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fc5e66f25ff411090cd8bb264a2451f" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:23.197804  2521 catalog_manager.cc:5697] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 1 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.207271  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864" is_pre_election: true
I20260709 12:22:23.213019  2771 raft_consensus.cc:3060] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.213335  2772 raft_consensus.cc:2468] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 0.
I20260709 12:22:23.211745  2620 raft_consensus.cc:3060] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.215503  2814 consensus_queue.cc:260] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.196996  2812 raft_consensus.cc:385] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.218256  2812 raft_consensus.cc:740] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.220827  2814 ts_tablet_manager.cc:1434] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:22:23.220528  2663 leader_election.cc:304] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 0cf90e155681407ea8ab2905f6bd31b0
I20260709 12:22:23.221657  2771 raft_consensus.cc:2468] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 1.
I20260709 12:22:23.222038  2814 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:23.223805  2663 leader_election.cc:304] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.223585  2812 consensus_queue.cc:260] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.225112  2821 raft_consensus.cc:2804] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.225561  2821 raft_consensus.cc:493] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.226089  2821 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.226227  2812 ts_tablet_manager.cc:1434] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:22:23.233487  2819 raft_consensus.cc:2804] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.247684  2819 raft_consensus.cc:697] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:23.248070  2812 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:23.248904  2819 consensus_queue.cc:237] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.226432  2846 raft_consensus.cc:493] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.250795  2846 raft_consensus.cc:515] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.253177  2846 leader_election.cc:290] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.254537  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0b5e339cec42f699083014deb0f51a" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.255311  2698 raft_consensus.cc:2468] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.255298  2622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0b5e339cec42f699083014deb0f51a" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.256326  2622 raft_consensus.cc:2468] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.257833  2736 leader_election.cc:304] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.258993  2812 tablet_bootstrap.cc:654] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.258739  2821 raft_consensus.cc:515] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.259342  2846 raft_consensus.cc:2804] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.259866  2846 raft_consensus.cc:493] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.260226  2846 raft_consensus.cc:3060] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.238977  2814 tablet_bootstrap.cc:654] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.228929  2813 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:23.261067  2813 ts_tablet_manager.cc:1403] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.185s	user 0.018s	sys 0.004s
I20260709 12:22:23.229115  2818 raft_consensus.cc:493] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.261963  2818 raft_consensus.cc:515] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.241961  2620 raft_consensus.cc:2468] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 1.
I20260709 12:22:23.264485  2813 raft_consensus.cc:359] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.266332  2813 raft_consensus.cc:385] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.228137  2621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.268447  2818 leader_election.cc:290] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.268970  2846 raft_consensus.cc:515] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.269265  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.266651  2813 raft_consensus.cc:740] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
W20260709 12:22:23.272548  2736 leader_election.cc:343] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:23.274675  2698 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.273602  2621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.275722  2621 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.276515  2738 leader_election.cc:304] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.277506  2818 raft_consensus.cc:2804] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.278012  2818 raft_consensus.cc:493] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.278411  2818 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.279660  2521 catalog_manager.cc:5697] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:23.280834  2621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0b5e339cec42f699083014deb0f51a" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.281117  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f0b5e339cec42f699083014deb0f51a" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:23.281929  2698 raft_consensus.cc:3060] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.282243  2813 consensus_queue.cc:260] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.281435  2621 raft_consensus.cc:3060] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.287124  2813 ts_tablet_manager.cc:1434] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.002s
I20260709 12:22:23.288486  2813 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:23.284487  2846 leader_election.cc:290] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.290820  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.270853  2812 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:23.295588  2620 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.300985  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:23.293167  2698 raft_consensus.cc:2468] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.301723  2813 tablet_bootstrap.cc:654] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.293588  2814 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:23.302682  2738 leader_election.cc:304] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.294718  2819 consensus_queue.cc:1048] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:23.303434  2814 ts_tablet_manager.cc:1403] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.082s	user 0.018s	sys 0.002s
I20260709 12:22:23.305691  2621 raft_consensus.cc:2468] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.295710  2812 ts_tablet_manager.cc:1403] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.008s
I20260709 12:22:23.308084  2846 raft_consensus.cc:2804] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.309564  2812 raft_consensus.cc:359] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.310241  2812 raft_consensus.cc:385] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.310513  2812 raft_consensus.cc:740] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.311203  2812 consensus_queue.cc:260] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.308799  2518 catalog_manager.cc:5697] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 1 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.316903  2771 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.320457  2821 leader_election.cc:290] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:23.328145  2814 raft_consensus.cc:359] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.329077  2814 raft_consensus.cc:385] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.329380  2814 raft_consensus.cc:740] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.330500  2620 raft_consensus.cc:2468] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 1.
I20260709 12:22:23.332231  2662 leader_election.cc:304] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: 
I20260709 12:22:23.325342  2818 raft_consensus.cc:515] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.335443  2821 raft_consensus.cc:2804] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.338761  2821 raft_consensus.cc:697] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:23.340003  2821 consensus_queue.cc:237] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.344110  2812 ts_tablet_manager.cc:1434] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.013s
I20260709 12:22:23.333887  2846 raft_consensus.cc:697] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:23.342137  2814 consensus_queue.cc:260] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.349253  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.343192  2860 raft_consensus.cc:493] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.349953  2620 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.352334  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:23.352986  2698 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.356117  2814 ts_tablet_manager.cc:1434] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.004s
I20260709 12:22:23.357288  2814 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:23.350270  2860 raft_consensus.cc:515] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.348786  2812 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864: Bootstrap starting.
I20260709 12:22:23.359401  2620 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.363765  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4511dc9b60d046a49fa1c3a1234cdc29" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.364629  2697 raft_consensus.cc:2468] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.365173  2736 leader_election.cc:304] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.366401  2862 raft_consensus.cc:2804] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.360821  2860 leader_election.cc:290] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.367039  2862 raft_consensus.cc:697] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:23.362234  2621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4511dc9b60d046a49fa1c3a1234cdc29" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.368170  2621 raft_consensus.cc:2468] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.368098  2862 consensus_queue.cc:237] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.372211  2812 tablet_bootstrap.cc:654] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.372964  2738 leader_election.cc:304] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.373138  2846 consensus_queue.cc:237] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.374696  2860 raft_consensus.cc:2804] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.375166  2860 raft_consensus.cc:493] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.375648  2860 raft_consensus.cc:3060] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.365144  2814 tablet_bootstrap.cc:654] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.340926  2818 leader_election.cc:290] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.381567  2771 raft_consensus.cc:2468] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 1.
I20260709 12:22:23.387624  2813 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:23.388360  2813 ts_tablet_manager.cc:1403] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.100s	user 0.028s	sys 0.019s
I20260709 12:22:23.391875  2698 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.394824  2813 raft_consensus.cc:359] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.395454  2813 raft_consensus.cc:385] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.395694  2813 raft_consensus.cc:740] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.396356  2813 consensus_queue.cc:260] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.397998  2813 ts_tablet_manager.cc:1434] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 12:22:23.398970  2813 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:23.404677  2813 tablet_bootstrap.cc:654] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.406024  2860 raft_consensus.cc:515] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.408658  2812 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864: No bootstrap required, opened a new log
I20260709 12:22:23.409332  2812 ts_tablet_manager.cc:1403] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.003s
I20260709 12:22:23.411597  2621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4511dc9b60d046a49fa1c3a1234cdc29" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.412557  2621 raft_consensus.cc:3060] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.412668  2812 raft_consensus.cc:359] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.413694  2812 raft_consensus.cc:385] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.414029  2812 raft_consensus.cc:740] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.419867  2812 consensus_queue.cc:260] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.422530  2814 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:23.423133  2814 ts_tablet_manager.cc:1403] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.005s
I20260709 12:22:23.424206  2812 ts_tablet_manager.cc:1434] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:22:23.425691  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4511dc9b60d046a49fa1c3a1234cdc29" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:23.427038  2698 raft_consensus.cc:3060] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.439538  2814 raft_consensus.cc:359] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.440454  2814 raft_consensus.cc:385] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.440723  2814 raft_consensus.cc:740] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.441694  2814 consensus_queue.cc:260] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.441247  2521 catalog_manager.cc:5697] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 1 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.442647  2620 raft_consensus.cc:3060] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.444790  2814 ts_tablet_manager.cc:1434] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:22:23.447638  2814 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:23.450024  2518 catalog_manager.cc:5697] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:23.464934  2821 consensus_queue.cc:1048] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:23.456606  2860 leader_election.cc:290] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.468384  2698 raft_consensus.cc:2468] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.455969  2814 tablet_bootstrap.cc:654] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.459640  2813 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:23.471959  2813 ts_tablet_manager.cc:1403] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.073s	user 0.023s	sys 0.029s
I20260709 12:22:23.472509  2621 raft_consensus.cc:2468] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.473765  2736 leader_election.cc:304] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.474665  2860 raft_consensus.cc:2804] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.475144  2860 raft_consensus.cc:697] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:23.475984  2860 consensus_queue.cc:237] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.475857  2813 raft_consensus.cc:359] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.479159  2813 raft_consensus.cc:385] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.479494  2813 raft_consensus.cc:740] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.480391  2813 consensus_queue.cc:260] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.482535  2813 ts_tablet_manager.cc:1434] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:22:23.483480  2813 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae: Bootstrap starting.
I20260709 12:22:23.491704  2813 tablet_bootstrap.cc:654] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.496531  2814 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:23.496994  2814 ts_tablet_manager.cc:1403] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.009s
I20260709 12:22:23.499034  2814 raft_consensus.cc:359] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.499743  2814 raft_consensus.cc:385] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.500027  2814 raft_consensus.cc:740] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.500700  2814 consensus_queue.cc:260] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.501363  2813 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae: No bootstrap required, opened a new log
I20260709 12:22:23.501900  2813 ts_tablet_manager.cc:1403] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.004s
I20260709 12:22:23.502715  2814 ts_tablet_manager.cc:1434] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:22:23.503667  2814 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:23.504722  2813 raft_consensus.cc:359] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.505472  2813 raft_consensus.cc:385] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.505776  2813 raft_consensus.cc:740] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.506639  2813 consensus_queue.cc:260] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.508880  2813 ts_tablet_manager.cc:1434] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:23.509810  2814 tablet_bootstrap.cc:654] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:23.509526  2518 catalog_manager.cc:5697] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:23.511945  2518 catalog_manager.cc:5697] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:23.520804  2814 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: No bootstrap required, opened a new log
I20260709 12:22:23.522083  2814 ts_tablet_manager.cc:1403] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:22:23.523002  2860 raft_consensus.cc:493] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.523588  2860 raft_consensus.cc:515] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.524955  2814 raft_consensus.cc:359] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.525727  2814 raft_consensus.cc:385] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:23.525993  2814 raft_consensus.cc:740] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:23.526747  2814 consensus_queue.cc:260] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.527858  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1507d1bea1424ff89b079b67055e8896" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.528228  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1507d1bea1424ff89b079b67055e8896" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.528792  2698 raft_consensus.cc:2468] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.529325  2814 ts_tablet_manager.cc:1434] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:23.529044  2620 raft_consensus.cc:2468] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.527114  2860 leader_election.cc:290] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.531032  2738 leader_election.cc:304] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.532176  2860 raft_consensus.cc:2804] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.532485  2860 raft_consensus.cc:493] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.532784  2860 raft_consensus.cc:3060] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.538977  2860 raft_consensus.cc:515] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.541384  2860 leader_election.cc:290] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.542002  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1507d1bea1424ff89b079b67055e8896" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.542152  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1507d1bea1424ff89b079b67055e8896" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:23.542840  2620 raft_consensus.cc:3060] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.543006  2698 raft_consensus.cc:3060] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.549254  2620 raft_consensus.cc:2468] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.549638  2698 raft_consensus.cc:2468] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.550379  2736 leader_election.cc:304] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.551321  2860 raft_consensus.cc:2804] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.551764  2860 raft_consensus.cc:697] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:23.552135  2862 raft_consensus.cc:493] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.552492  2860 consensus_queue.cc:237] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.552719  2862 raft_consensus.cc:515] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.554832  2862 leader_election.cc:290] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.555745  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0" is_pre_election: true
I20260709 12:22:23.556587  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.556680  2620 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.557724  2698 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 0.
I20260709 12:22:23.558635  2736 leader_election.cc:304] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.559780  2860 raft_consensus.cc:2804] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.560259  2860 raft_consensus.cc:493] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.560551  2860 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.562242  2519 catalog_manager.cc:5697] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:23.566926  2860 raft_consensus.cc:515] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.569036  2860 leader_election.cc:290] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:23.569959  2620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:23.570731  2620 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.570600  2698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:23.571352  2698 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.578907  2698 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.578907  2620 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:23.580485  2738 leader_election.cc:304] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.581298  2860 raft_consensus.cc:2804] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.581799  2860 raft_consensus.cc:697] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:23.582463  2860 consensus_queue.cc:237] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.593185  2519 catalog_manager.cc:5697] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:23.605422  2860 consensus_queue.cc:1048] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:23.611822  2820 raft_consensus.cc:493] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.612478  2820 raft_consensus.cc:515] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.614523  2820 leader_election.cc:290] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:23.616168  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864" is_pre_election: true
I20260709 12:22:23.618857  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.620033  2620 raft_consensus.cc:3060] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.620682  2589 leader_election.cc:304] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0; no voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864
I20260709 12:22:23.629128  2820 raft_consensus.cc:2749] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:23.629963  2846 consensus_queue.cc:1048] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:23.644491  2820 raft_consensus.cc:493] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.645246  2820 raft_consensus.cc:515] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.647604  2820 leader_election.cc:290] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:23.648432  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:23.649166  2697 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cf90e155681407ea8ab2905f6bd31b0 in term 0.
I20260709 12:22:23.649093  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864" is_pre_election: true
I20260709 12:22:23.650462  2771 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cf90e155681407ea8ab2905f6bd31b0 in term 0.
I20260709 12:22:23.650702  2589 leader_election.cc:304] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: 
I20260709 12:22:23.651576  2820 raft_consensus.cc:2804] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:23.651949  2820 raft_consensus.cc:493] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.652277  2820 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.659437  2820 raft_consensus.cc:515] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.661028  2821 consensus_queue.cc:1048] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:23.661504  2820 leader_election.cc:290] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 election: Requested vote from peers 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:23.662832  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:23.662775  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "0cf90e155681407ea8ab2905f6bd31b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:23.663725  2697 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.663790  2771 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:23.672457  2697 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cf90e155681407ea8ab2905f6bd31b0 in term 1.
I20260709 12:22:23.672457  2771 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cf90e155681407ea8ab2905f6bd31b0 in term 1.
I20260709 12:22:23.674116  2588 leader_election.cc:304] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:23.675181  2820 raft_consensus.cc:2804] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:23.678102  2819 consensus_queue.cc:1048] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:23.678148  2820 raft_consensus.cc:697] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 LEADER]: Becoming Leader. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Running, Role: LEADER
I20260709 12:22:23.680956  2820 consensus_queue.cc:237] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:23.693259  2521 catalog_manager.cc:5697] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "0cf90e155681407ea8ab2905f6bd31b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:23.750589  2846 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:23.762877  2846 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:23.807639  2846 consensus_queue.cc:1048] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:23.818943  2846 consensus_queue.cc:1048] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:23.856771 24421 ts_manager.cc:295] Set tserver state for 0cf90e155681407ea8ab2905f6bd31b0 to MAINTENANCE_MODE
I20260709 12:22:23.857828 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
I20260709 12:22:23.860394  2819 consensus_queue.cc:1048] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:23.871536  2662 consensus_peers.cc:597] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:23.880656  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:23.892032 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:22:23.892184  2662 consensus_peers.cc:597] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:23.893666 24421 tablet_replica.cc:333] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.895056 24421 raft_consensus.cc:2243] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:23.895985 24421 pending_rounds.cc:70] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Trying to abort 1 pending ops.
I20260709 12:22:23.896215 24421 pending_rounds.cc:77] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305624550129762304 op_type: NO_OP noop_request { }
I20260709 12:22:23.896569 24421 raft_consensus.cc:2889] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:22:23.896869 24421 raft_consensus.cc:2272] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.899324 24421 tablet_replica.cc:333] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.899888 24421 raft_consensus.cc:2243] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.900296 24421 raft_consensus.cc:2272] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.902642 24421 tablet_replica.cc:333] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.903173 24421 raft_consensus.cc:2243] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.903540 24421 raft_consensus.cc:2272] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.905831 24421 tablet_replica.cc:333] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.906850 24421 raft_consensus.cc:2243] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.907362 24421 raft_consensus.cc:2272] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.909798 24421 tablet_replica.cc:333] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.910327 24421 raft_consensus.cc:2243] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.910812 24421 raft_consensus.cc:2272] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.912743 24421 tablet_replica.cc:333] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.913266 24421 raft_consensus.cc:2243] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.913636 24421 raft_consensus.cc:2272] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.915393 24421 tablet_replica.cc:333] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.915922 24421 raft_consensus.cc:2243] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.916324 24421 raft_consensus.cc:2272] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.918644 24421 tablet_replica.cc:333] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.919189 24421 raft_consensus.cc:2243] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.919579 24421 raft_consensus.cc:2272] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.922062 24421 tablet_replica.cc:333] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.922986 24421 raft_consensus.cc:2243] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.970831 24421 raft_consensus.cc:2272] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:23.986864  2821 consensus_queue.cc:1048] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:22:23.989032 24421 tablet_replica.cc:333] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: stopping tablet replica
I20260709 12:22:23.988545  2846 consensus_queue.cc:1048] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:23.990651 24421 raft_consensus.cc:2243] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:23.991905 24421 raft_consensus.cc:2272] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:24.036803  2736 consensus_peers.cc:597] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:24.041738  2846 consensus_queue.cc:1048] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:24.042824  2736 consensus_peers.cc:597] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:24.066424 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
W20260709 12:22:24.082551  2736 consensus_peers.cc:597] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:24.102209  2662 consensus_peers.cc:597] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:24.114180  2719 debug-util.cc:398] Leaking SignalData structure 0x7b080000b8c0 after lost signal to thread 24424
W20260709 12:22:24.116484  2719 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8600 after lost signal to thread 2555
I20260709 12:22:24.116694  2860 consensus_queue.cc:1048] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:22:24.121037  2719 debug-util.cc:398] Leaking SignalData structure 0x7b080009e0a0 after lost signal to thread 2722
W20260709 12:22:24.122948  2719 debug-util.cc:398] Leaking SignalData structure 0x7b080029b760 after lost signal to thread 2796
W20260709 12:22:24.141896  2736 consensus_peers.cc:597] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:24.341836  2736 consensus_peers.cc:597] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:24.426667  2736 consensus_peers.cc:597] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:24.882268  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:25.064544  2662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111) [suppressed 126 similar messages]
I20260709 12:22:25.387389  2862 raft_consensus.cc:493] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.388384  2862 raft_consensus.cc:515] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:25.392860  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" is_pre_election: true
I20260709 12:22:25.393992  2697 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 1.
I20260709 12:22:25.398234  2862 leader_election.cc:290] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:25.404331  2738 leader_election.cc:304] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:25.405710  2862 raft_consensus.cc:2804] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:25.406109  2862 raft_consensus.cc:493] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.406472  2862 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:25.408578  2736 leader_election.cc:336] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111)
I20260709 12:22:25.419773  2862 raft_consensus.cc:515] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:25.423136  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:25.423802  2697 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:25.427125  2736 leader_election.cc:336] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111)
I20260709 12:22:25.434291  2697 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 2.
I20260709 12:22:25.435676  2862 leader_election.cc:290] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:25.436070  2738 leader_election.cc:304] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 0cf90e155681407ea8ab2905f6bd31b0
I20260709 12:22:25.438289  2862 raft_consensus.cc:2804] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:25.439507  2862 raft_consensus.cc:697] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:25.440924  2862 consensus_queue.cc:237] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:25.643035  2521 catalog_manager.cc:5697] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 1 to 2, leader changed from 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65) to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
W20260709 12:22:25.883599  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:25.886416  2697 raft_consensus.cc:1275] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:22:25.888016  2862 consensus_queue.cc:1048] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:22:25.907408  2736 consensus_peers.cc:597] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:26.287905  2662 consensus_peers.cc:597] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.339792  2736 consensus_peers.cc:597] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.370579  2736 consensus_peers.cc:597] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.403472  2736 consensus_peers.cc:597] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.483345  2552 debug-util.cc:398] Leaking SignalData structure 0x7b080005cce0 after lost signal to thread 24424
W20260709 12:22:26.484784  2552 debug-util.cc:398] Leaking SignalData structure 0x7b0800341fc0 after lost signal to thread 2555
W20260709 12:22:26.486093  2552 debug-util.cc:398] Leaking SignalData structure 0x7b08002c5180 after lost signal to thread 2722
W20260709 12:22:26.487286  2552 debug-util.cc:398] Leaking SignalData structure 0x7b0800099d00 after lost signal to thread 2796
W20260709 12:22:26.708678  2662 consensus_peers.cc:597] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.720281  2736 consensus_peers.cc:597] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.746752  2662 consensus_peers.cc:597] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.852924  2736 consensus_peers.cc:597] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:26.884985  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:27.141076  2736 consensus_peers.cc:597] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:27.886523  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:28.331789  2736 consensus_peers.cc:597] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:28.597867  2736 consensus_peers.cc:597] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:28.717806  2662 consensus_peers.cc:597] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:28.887820  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:22:28.992372  2736 consensus_peers.cc:597] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:29.016151  2736 consensus_peers.cc:597] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:29.127887 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:29.145291  2913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.147629  2662 consensus_peers.cc:597] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:29.152599  2914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:29.154968  2916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:29.157006 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:29.159868 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:29.160132 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:29.160332 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599749160313 us; error 0 us; skew 500 ppm
I20260709 12:22:29.160895 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:29.164377 24421 webserver.cc:533] Webserver started at http://127.23.217.65:36555/ using document root <none> and password file <none>
I20260709 12:22:29.164912 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:29.165125 24421 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:22:29.169915 24421 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:29.174790  2921 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:29.175716 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:29.176101 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "0cf90e155681407ea8ab2905f6bd31b0"
format_stamp: "Formatted at 2026-07-09 12:22:22 on dist-test-slave-lg8f"
I20260709 12:22:29.176506 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:29.193471 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:29.194937 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:29.197019 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:29.202025  2928 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:22:29.261639 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:22:29.262141 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.062s	user 0.002s	sys 0.001s
I20260709 12:22:29.262672 24421 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:22:29.272487  2928 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.284619  2928 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.285331  2928 tablet_bootstrap.cc:492] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.285921  2928 ts_tablet_manager.cc:1403] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 12:22:29.288223  2928 raft_consensus.cc:359] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.288789  2928 raft_consensus.cc:740] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.289475  2928 consensus_queue.cc:260] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.292071  2928 ts_tablet_manager.cc:1434] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.008s
I20260709 12:22:29.292762  2928 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
W20260709 12:22:29.300665  2736 consensus_peers.cc:597] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:29.306241  2928 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.307369  2928 tablet_bootstrap.cc:492] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.308094  2928 ts_tablet_manager.cc:1403] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:22:29.310549  2928 raft_consensus.cc:359] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.311316  2928 raft_consensus.cc:740] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.312109  2928 consensus_queue.cc:260] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.313920  2928 ts_tablet_manager.cc:1434] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:29.314764  2928 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.328642  2928 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.329734  2928 tablet_bootstrap.cc:492] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.330461  2928 ts_tablet_manager.cc:1403] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 12:22:29.333336  2928 raft_consensus.cc:359] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.333896  2928 raft_consensus.cc:740] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.334697  2928 consensus_queue.cc:260] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.336355  2928 ts_tablet_manager.cc:1434] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.002s
I20260709 12:22:29.337067  2928 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.347725  2928 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.348399  2928 tablet_bootstrap.cc:492] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.348737 24421 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:22:29.348927  2928 ts_tablet_manager.cc:1403] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:22:29.349228 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.087s	user 0.068s	sys 0.007s
I20260709 12:22:29.351445  2928 raft_consensus.cc:359] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.352064  2928 raft_consensus.cc:740] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.352753  2928 consensus_queue.cc:260] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.354403  2928 ts_tablet_manager.cc:1434] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:22:29.355083  2928 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.372757  2928 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:22:29.373799  2928 tablet_bootstrap.cc:492] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.374426  2928 ts_tablet_manager.cc:1403] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.001s
I20260709 12:22:29.377235  2928 raft_consensus.cc:359] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.378119  2928 raft_consensus.cc:740] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.378952  2928 consensus_queue.cc:260] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.381116  2928 ts_tablet_manager.cc:1434] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:29.382027  2928 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.395047  2928 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.395965  2928 tablet_bootstrap.cc:492] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.396490  2928 ts_tablet_manager.cc:1403] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 12:22:29.398862  2928 raft_consensus.cc:359] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.399564  2928 raft_consensus.cc:740] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.400374  2928 consensus_queue.cc:260] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.402127  2928 ts_tablet_manager.cc:1434] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:22:29.402827  2928 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.417448  2928 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.419100  2928 tablet_bootstrap.cc:492] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.419734  2928 ts_tablet_manager.cc:1403] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.008s
I20260709 12:22:29.422242  2928 raft_consensus.cc:359] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.423041  2928 raft_consensus.cc:740] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.423980  2928 consensus_queue.cc:260] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.427335  2928 ts_tablet_manager.cc:1434] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:29.428406  2928 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
W20260709 12:22:29.439502  2662 consensus_peers.cc:597] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:29.449487  2928 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.450541  2928 tablet_bootstrap.cc:492] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.451341  2928 ts_tablet_manager.cc:1403] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.007s
I20260709 12:22:29.453486  2928 raft_consensus.cc:359] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.454262  2928 raft_consensus.cc:740] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.455264  2928 consensus_queue.cc:260] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.457059  2928 ts_tablet_manager.cc:1434] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:29.457670  2928 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.472290  2928 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.473299  2928 tablet_bootstrap.cc:492] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.473896  2928 ts_tablet_manager.cc:1403] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.002s
I20260709 12:22:29.476315  2928 raft_consensus.cc:359] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.476974  2928 raft_consensus.cc:740] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.477811 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:34889
I20260709 12:22:29.477730  2928 consensus_queue.cc:260] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.478039  2994 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:34889 every 8 connection(s)
I20260709 12:22:29.480585  2928 ts_tablet_manager.cc:1434] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.001s
I20260709 12:22:29.481369  2928 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap starting.
I20260709 12:22:29.497936 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:29.518872  2928 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:29.520361  2928 tablet_bootstrap.cc:492] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Bootstrap complete.
I20260709 12:22:29.521409  2928 ts_tablet_manager.cc:1403] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.011s
I20260709 12:22:29.524475  2928 raft_consensus.cc:359] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:29.525349  2928 raft_consensus.cc:740] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf90e155681407ea8ab2905f6bd31b0, State: Initialized, Role: FOLLOWER
I20260709 12:22:29.526257  2928 consensus_queue.cc:260] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } } peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } }
I20260709 12:22:29.528329  2928 ts_tablet_manager.cc:1434] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:29.535192  2995 heartbeater.cc:344] Connected to a master server at 127.23.217.126:42545
I20260709 12:22:29.535817  2995 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:29.536834  2995 heartbeater.cc:507] Master 127.23.217.126:42545 requested a full tablet report, sending...
I20260709 12:22:29.540346  2772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99"
dest_uuid: "e9352e7947904c43a959a977679d8864"
mode: GRACEFUL
new_leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:45506
I20260709 12:22:29.541268  2772 raft_consensus.cc:606] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Received request to transfer leadership to TS 0cb4525738614d1a8b56c5af1d7a7cae
I20260709 12:22:29.548611  2772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4"
dest_uuid: "e9352e7947904c43a959a977679d8864"
mode: GRACEFUL
new_leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:45506
I20260709 12:22:29.549304  2772 raft_consensus.cc:606] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Received request to transfer leadership to TS 0cb4525738614d1a8b56c5af1d7a7cae
I20260709 12:22:29.549582  2519 ts_manager.cc:194] Re-registered known tserver with Master: 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889)
I20260709 12:22:29.554057  2772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a737173da1a44edf98cb7483d880e113"
dest_uuid: "e9352e7947904c43a959a977679d8864"
mode: GRACEFUL
new_leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:45506
I20260709 12:22:29.555068  2772 raft_consensus.cc:606] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Received request to transfer leadership to TS 0cb4525738614d1a8b56c5af1d7a7cae
I20260709 12:22:29.557301 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:22:29.558291 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:29.559566 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:29.568120  2519 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38586
I20260709 12:22:29.574997  2995 heartbeater.cc:499] Master 127.23.217.126:42545 was elected leader, sending a full tablet report...
I20260709 12:22:29.576941  3003 raft_consensus.cc:993] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: : Instructing follower 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:29.581458  3003 raft_consensus.cc:1081] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Signalling peer 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:29.584611  2697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:48758
I20260709 12:22:29.585440  2697 raft_consensus.cc:493] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:29.585872  2697 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.596936  2697 raft_consensus.cc:515] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.599251  2697 leader_election.cc:290] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 2 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:29.601931  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:29.603139  2772 raft_consensus.cc:3055] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:29.603618  2772 raft_consensus.cc:740] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:29.606721  2772 consensus_queue.cc:260] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.608320  2772 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.617151  2966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:29.618386  2966 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.618225  2772 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 2.
I20260709 12:22:29.624958  2663 leader_election.cc:304] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:29.626050  2900 raft_consensus.cc:2804] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:29.644450  2966 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 2.
I20260709 12:22:29.645834  2900 raft_consensus.cc:697] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:29.647548  2900 consensus_queue.cc:237] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.663791  2519 catalog_manager.cc:5697] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 1 to 2, leader changed from e9352e7947904c43a959a977679d8864 (127.23.217.67) to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 2 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:29.751289  2966 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.760794  2966 pending_rounds.cc:85] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Aborting all ops after (but not including) 0
I20260709 12:22:29.761143  2966 pending_rounds.cc:107] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:22:29.761480  2966 raft_consensus.cc:2889] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:29.782457  3003 raft_consensus.cc:993] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: : Instructing follower 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:29.783197  3004 raft_consensus.cc:1081] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Signalling peer 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:29.784937  2697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a737173da1a44edf98cb7483d880e113"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:48758
I20260709 12:22:29.785627  2697 raft_consensus.cc:493] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:29.786029  2697 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.795418  2697 raft_consensus.cc:515] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.798327  2697 leader_election.cc:290] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 2 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:29.798969  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:29.799741  2772 raft_consensus.cc:3055] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:29.798840  2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:29.800143  2772 raft_consensus.cc:740] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:29.800529  2967 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.801280  2772 consensus_queue.cc:260] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.802386  2772 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.808336  2772 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 2.
I20260709 12:22:29.809356  2663 leader_election.cc:304] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:29.810083  2967 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 2.
I20260709 12:22:29.810086  2900 raft_consensus.cc:2804] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:29.811077  2900 raft_consensus.cc:697] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:29.812114  2900 consensus_queue.cc:237] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.821265  2519 catalog_manager.cc:5697] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 1 to 2, leader changed from e9352e7947904c43a959a977679d8864 (127.23.217.67) to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 2 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:29.852532  3003 raft_consensus.cc:993] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864: : Instructing follower 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:29.853075  3004 raft_consensus.cc:1081] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Signalling peer 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:29.854988  2697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:48758
I20260709 12:22:29.855626  2697 raft_consensus.cc:493] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:29.855988  2697 raft_consensus.cc:3060] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.862251  2697 raft_consensus.cc:515] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.863929  2697 leader_election.cc:290] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 2 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:29.865238  2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:29.865299  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f72a8b042f84bf0b1691128bac6b0f4" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:29.866133  2967 raft_consensus.cc:3060] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.866257  2772 raft_consensus.cc:3055] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:29.866816  2772 raft_consensus.cc:740] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:29.867708  2772 consensus_queue.cc:260] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.868914  2772 raft_consensus.cc:3060] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.874809  2967 raft_consensus.cc:2468] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 2.
I20260709 12:22:29.876310  2662 leader_election.cc:304] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: 
I20260709 12:22:29.876816  2772 raft_consensus.cc:2468] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 2.
I20260709 12:22:29.877444  3006 raft_consensus.cc:2804] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:29.878075  3006 raft_consensus.cc:697] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:29.879168  3006 consensus_queue.cc:237] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:29.888382  2519 catalog_manager.cc:5697] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 1 to 2, leader changed from e9352e7947904c43a959a977679d8864 (127.23.217.67) to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 2 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:29.889127  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:30.198403  2772 raft_consensus.cc:1275] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:30.200070  3006 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:30.215693  2967 raft_consensus.cc:1275] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:30.221339  3006 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:30.251682  2772 raft_consensus.cc:1275] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:30.253100  3006 consensus_queue.cc:1048] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:30.260272  2967 raft_consensus.cc:1275] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:30.261902  3021 consensus_queue.cc:1048] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:30.293552  2967 raft_consensus.cc:1275] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:30.294994  3021 consensus_queue.cc:1048] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:30.306051  2772 raft_consensus.cc:1275] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:30.307857  2900 consensus_queue.cc:1048] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:30.581717 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:30.603245  2697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
mode: GRACEFUL
new_leader_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:45584
I20260709 12:22:30.603751  2697 raft_consensus.cc:606] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Received request to transfer leadership to TS e9352e7947904c43a959a977679d8864
I20260709 12:22:30.606205  2697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a737173da1a44edf98cb7483d880e113"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
mode: GRACEFUL
new_leader_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:45584
I20260709 12:22:30.606655  2697 raft_consensus.cc:606] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Received request to transfer leadership to TS e9352e7947904c43a959a977679d8864
I20260709 12:22:30.607692 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:30.608250 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:30.608903 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:30.679793  3006 raft_consensus.cc:993] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae: : Instructing follower e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:30.680429  2900 raft_consensus.cc:1081] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Signalling peer e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:30.682173  2772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a737173da1a44edf98cb7483d880e113"
dest_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:56744
I20260709 12:22:30.682734  2772 raft_consensus.cc:493] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:30.683027  2772 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.688984  2772 raft_consensus.cc:515] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:30.690907  2772 leader_election.cc:290] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 3 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:30.691952  2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:30.692155  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:30.692759  2967 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.692867  2697 raft_consensus.cc:3055] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:30.693354  2697 raft_consensus.cc:740] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:30.694052  2697 consensus_queue.cc:260] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:30.694994  2697 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.698918  2967 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 3.
I20260709 12:22:30.699903  2697 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 3.
I20260709 12:22:30.700403  2736 leader_election.cc:304] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:30.701401  2912 raft_consensus.cc:2804] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:30.701861  2912 raft_consensus.cc:697] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:30.702556  2912 consensus_queue.cc:237] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:30.708244  2521 catalog_manager.cc:5697] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 2 to 3, leader changed from 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66) to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:30.873387  3006 raft_consensus.cc:993] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae: : Instructing follower e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:30.873981  3006 raft_consensus.cc:1081] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Signalling peer e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:30.875685  2772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99"
dest_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:56744
I20260709 12:22:30.876262  2772 raft_consensus.cc:493] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:30.876588  2772 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.882813  2772 raft_consensus.cc:515] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:30.884681  2772 leader_election.cc:290] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 3 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:30.885809  2966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:30.885852  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:30.886546  2966 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.886755  2697 raft_consensus.cc:3055] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:30.887135  2697 raft_consensus.cc:740] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:30.887733  2697 consensus_queue.cc:260] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:30.888609  2697 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:30.890439  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:30.893329  2966 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 3.
I20260709 12:22:30.894702  2736 leader_election.cc:304] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:30.895102  2697 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 3.
I20260709 12:22:30.895619  2912 raft_consensus.cc:2804] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:30.896189  2912 raft_consensus.cc:697] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:30.897006  2912 consensus_queue.cc:237] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:30.903235  2521 catalog_manager.cc:5697] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 2 to 3, leader changed from 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66) to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:31.138815  2697 raft_consensus.cc:1275] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.140213  2912 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.152220  2966 raft_consensus.cc:1275] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.154011  2912 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.343968  2697 raft_consensus.cc:1275] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.345551  2912 consensus_queue.cc:1048] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.354602  2967 raft_consensus.cc:1275] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.355990  3004 consensus_queue.cc:1048] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.615540 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:31.624264  2772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a737173da1a44edf98cb7483d880e113"
dest_uuid: "e9352e7947904c43a959a977679d8864"
mode: GRACEFUL
new_leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:45506
I20260709 12:22:31.624850  2772 raft_consensus.cc:606] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Received request to transfer leadership to TS 0cb4525738614d1a8b56c5af1d7a7cae
I20260709 12:22:31.627470  2772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e42c4862903745279ec09f631252173d"
dest_uuid: "e9352e7947904c43a959a977679d8864"
mode: GRACEFUL
new_leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:45506
I20260709 12:22:31.628041  2772 raft_consensus.cc:606] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Received request to transfer leadership to TS 0cb4525738614d1a8b56c5af1d7a7cae
I20260709 12:22:31.629195 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:31.629752 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:31.630316 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:31.785506  3004 raft_consensus.cc:993] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864: : Instructing follower 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:31.786074  3038 raft_consensus.cc:1081] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Signalling peer 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:31.787524  2697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e42c4862903745279ec09f631252173d"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:48758
I20260709 12:22:31.788092  2697 raft_consensus.cc:493] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.788354  2697 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.793852  2697 raft_consensus.cc:515] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:31.795703  2697 leader_election.cc:290] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 3 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:31.796811  2966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:31.796891  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:31.797505  2966 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.797713  2772 raft_consensus.cc:3055] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:31.798163  2772 raft_consensus.cc:740] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:31.799072  2772 consensus_queue.cc:260] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:31.800160  2772 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.803175  2966 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 3.
I20260709 12:22:31.804361  2662 leader_election.cc:304] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: 
I20260709 12:22:31.805152  3006 raft_consensus.cc:2804] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.806335  3006 raft_consensus.cc:697] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:31.806964  2772 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 3.
I20260709 12:22:31.807513  3006 consensus_queue.cc:237] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:31.815897  2518 catalog_manager.cc:5697] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 2 to 3, leader changed from e9352e7947904c43a959a977679d8864 (127.23.217.67) to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 3 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:31.891744  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:31.931962  2912 raft_consensus.cc:993] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: : Instructing follower 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:31.932651  3038 raft_consensus.cc:1081] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Signalling peer 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:31.934218  2697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a737173da1a44edf98cb7483d880e113"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:48758
I20260709 12:22:31.934760  2697 raft_consensus.cc:493] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.935058  2697 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.940631  2697 raft_consensus.cc:515] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:31.942250  2697 leader_election.cc:290] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 4 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:31.943374  2966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:31.943457  2772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a737173da1a44edf98cb7483d880e113" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:31.944073  2966 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.944222  2772 raft_consensus.cc:3055] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:31.944689  2772 raft_consensus.cc:740] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:31.945446  2772 consensus_queue.cc:260] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:31.946583  2772 raft_consensus.cc:3060] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.950096  2966 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 4.
I20260709 12:22:31.951377  2662 leader_election.cc:304] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: 
I20260709 12:22:31.952338  3006 raft_consensus.cc:2804] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:31.952818  3006 raft_consensus.cc:697] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:31.953034  2772 raft_consensus.cc:2468] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 4.
I20260709 12:22:31.953691  3006 consensus_queue.cc:237] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:31.960381  2518 catalog_manager.cc:5697] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 3 to 4, leader changed from e9352e7947904c43a959a977679d8864 (127.23.217.67) to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 4 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:32.310549  2772 raft_consensus.cc:1275] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:32.312078  3006 consensus_queue.cc:1048] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:32.325273  2966 raft_consensus.cc:1275] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:32.327419  3040 consensus_queue.cc:1048] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:32.334944  2967 raft_consensus.cc:1275] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:32.336308  3040 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:32.351508  2772 raft_consensus.cc:1275] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:32.353206  3006 consensus_queue.cc:1048] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:32.636682 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:32.645328  2697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
mode: GRACEFUL
new_leader_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:45584
I20260709 12:22:32.645864  2697 raft_consensus.cc:606] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Received request to transfer leadership to TS e9352e7947904c43a959a977679d8864
I20260709 12:22:32.648836  2697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e42c4862903745279ec09f631252173d"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
mode: GRACEFUL
new_leader_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:45584
I20260709 12:22:32.649504  2697 raft_consensus.cc:606] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 LEADER]: Received request to transfer leadership to TS e9352e7947904c43a959a977679d8864
I20260709 12:22:32.650665 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:32.651316 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:32.651935 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:22:32.892948  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:32.897028  3047 raft_consensus.cc:993] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae: : Instructing follower e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:32.897619  3047 raft_consensus.cc:1081] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 LEADER]: Signalling peer e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:32.900146  2771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e42c4862903745279ec09f631252173d"
dest_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:56744
I20260709 12:22:32.900717  2771 raft_consensus.cc:493] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:32.901088  2771 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:32.907595  2771 raft_consensus.cc:515] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:32.909126  2771 leader_election.cc:290] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 4 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:32.910043  2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:32.910343  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42c4862903745279ec09f631252173d" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:32.910882  2967 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:32.910925  2697 raft_consensus.cc:3055] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:32.911501  2697 raft_consensus.cc:740] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:32.912359  2697 consensus_queue.cc:260] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:32.913638  2697 raft_consensus.cc:3060] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:32.919044  2967 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 4.
I20260709 12:22:32.920527  2736 leader_election.cc:304] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:32.921419  3038 raft_consensus.cc:2804] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:32.921855  2697 raft_consensus.cc:2468] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 4.
I20260709 12:22:32.924858  3038 raft_consensus.cc:697] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 4 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:32.926332  3038 consensus_queue.cc:237] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:32.937093  2518 catalog_manager.cc:5697] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 3 to 4, leader changed from 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66) to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 4 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.041239  3055 raft_consensus.cc:993] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae: : Instructing follower e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:33.041915  3047 raft_consensus.cc:1081] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Signalling peer e9352e7947904c43a959a977679d8864 to start an election
I20260709 12:22:33.043869  2771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da"
dest_uuid: "e9352e7947904c43a959a977679d8864"
 from {username='slave'} at 127.0.0.1:56744
I20260709 12:22:33.044435  2771 raft_consensus.cc:493] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:33.044783  2771 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.050753  2771 raft_consensus.cc:515] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:33.052649  2771 leader_election.cc:290] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357)
I20260709 12:22:33.053651  2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:33.053771  2697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "e9352e7947904c43a959a977679d8864" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
I20260709 12:22:33.054396  2967 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.054446  2697 raft_consensus.cc:3055] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:33.055127  2697 raft_consensus.cc:740] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:33.055789  2697 consensus_queue.cc:260] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:33.056829  2697 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.060972  2967 raft_consensus.cc:2468] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 2.
I20260709 12:22:33.061442  2697 raft_consensus.cc:2468] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9352e7947904c43a959a977679d8864 in term 2.
I20260709 12:22:33.062389  2736 leader_election.cc:304] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf90e155681407ea8ab2905f6bd31b0, e9352e7947904c43a959a977679d8864; no voters: 
I20260709 12:22:33.063243  3038 raft_consensus.cc:2804] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:33.063736  3038 raft_consensus.cc:697] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Becoming Leader. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:33.064563  3038 consensus_queue.cc:237] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:33.071255  2518 catalog_manager.cc:5697] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 reported cstate change: term changed from 1 to 2, leader changed from 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66) to e9352e7947904c43a959a977679d8864 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "e9352e7947904c43a959a977679d8864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.520170  2967 raft_consensus.cc:1275] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:33.521369  3038 consensus_queue.cc:1048] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:33.531813  2697 raft_consensus.cc:1275] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:33.533774  3038 consensus_queue.cc:1048] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:33.540748  2966 raft_consensus.cc:1275] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:33.542426  3066 consensus_queue.cc:1048] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:33.554033  2697 raft_consensus.cc:1275] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Refusing update from remote peer e9352e7947904c43a959a977679d8864: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:33.556195  3064 consensus_queue.cc:1048] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:33.658965 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:33.672039  2771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99"
dest_uuid: "e9352e7947904c43a959a977679d8864"
mode: GRACEFUL
new_leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:45506
I20260709 12:22:33.672605  2771 raft_consensus.cc:606] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Received request to transfer leadership to TS 0cb4525738614d1a8b56c5af1d7a7cae
I20260709 12:22:33.675555  2771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da"
dest_uuid: "e9352e7947904c43a959a977679d8864"
mode: GRACEFUL
new_leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:45506
I20260709 12:22:33.676273  2771 raft_consensus.cc:606] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Received request to transfer leadership to TS 0cb4525738614d1a8b56c5af1d7a7cae
I20260709 12:22:33.677675 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:33.678562 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:33.679385 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:33.888553  3038 raft_consensus.cc:993] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: : Instructing follower 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:33.889199  3064 raft_consensus.cc:1081] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Signalling peer 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:33.890969  2697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:48758
I20260709 12:22:33.891409  2697 raft_consensus.cc:493] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:33.891690  2697 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:33.894065  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:33.896694  2697 raft_consensus.cc:515] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:33.898286  2697 leader_election.cc:290] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 4 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:33.899279  2966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:33.899518  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb3b649044104bf06aa1fcd15cb99" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:33.899991  2966 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.900122  2771 raft_consensus.cc:3055] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:33.900565  2771 raft_consensus.cc:740] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:33.901238  2771 consensus_queue.cc:260] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:33.902222  2771 raft_consensus.cc:3060] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.906584  2966 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 4.
I20260709 12:22:33.907852  2771 raft_consensus.cc:2468] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 4.
I20260709 12:22:33.907887  2662 leader_election.cc:304] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: 
I20260709 12:22:33.909132  3047 raft_consensus.cc:2804] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:33.909682  3047 raft_consensus.cc:697] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:33.910480  3047 consensus_queue.cc:237] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:33.918081  2519 catalog_manager.cc:5697] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 3 to 4, leader changed from e9352e7947904c43a959a977679d8864 (127.23.217.67) to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 4 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.103253  3077 raft_consensus.cc:993] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864: : Instructing follower 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:34.105532  3064 raft_consensus.cc:1081] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Signalling peer 0cb4525738614d1a8b56c5af1d7a7cae to start an election
I20260709 12:22:34.107774  2697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da"
dest_uuid: "0cb4525738614d1a8b56c5af1d7a7cae"
 from {username='slave'} at 127.0.0.1:48758
I20260709 12:22:34.108844  2697 raft_consensus.cc:493] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.109387  2697 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.117712  2697 raft_consensus.cc:515] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:34.119781  2697 leader_election.cc:290] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 3 election: Requested vote from peers 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889), e9352e7947904c43a959a977679d8864 (127.23.217.67:34351)
I20260709 12:22:34.121321  2967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0cf90e155681407ea8ab2905f6bd31b0"
I20260709 12:22:34.121483  2771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdeb55b57c25429f9647fc81d00861da" candidate_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9352e7947904c43a959a977679d8864"
I20260709 12:22:34.122161  2967 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.122373  2771 raft_consensus.cc:3055] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.122951  2771 raft_consensus.cc:740] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9352e7947904c43a959a977679d8864, State: Running, Role: LEADER
I20260709 12:22:34.124399  2771 consensus_queue.cc:260] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:34.126688  2771 raft_consensus.cc:3060] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.131948  2967 raft_consensus.cc:2468] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 3.
I20260709 12:22:34.133718  2662 leader_election.cc:304] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb4525738614d1a8b56c5af1d7a7cae, 0cf90e155681407ea8ab2905f6bd31b0; no voters: 
I20260709 12:22:34.134965  3047 raft_consensus.cc:2804] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:34.135406  3047 raft_consensus.cc:697] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 LEADER]: Becoming Leader. State: Replica: 0cb4525738614d1a8b56c5af1d7a7cae, State: Running, Role: LEADER
I20260709 12:22:34.136657  3047 consensus_queue.cc:237] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } }
I20260709 12:22:34.137184  2771 raft_consensus.cc:2468] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cb4525738614d1a8b56c5af1d7a7cae in term 3.
I20260709 12:22:34.146433  2519 catalog_manager.cc:5697] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae reported cstate change: term changed from 2 to 3, leader changed from e9352e7947904c43a959a977679d8864 (127.23.217.67) to 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66). New cstate: current_term: 3 leader_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb4525738614d1a8b56c5af1d7a7cae" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.333380  2967 raft_consensus.cc:1275] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:34.335004  3047 consensus_queue.cc:1048] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:34.349203  2771 raft_consensus.cc:1275] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:34.351581  3071 consensus_queue.cc:1048] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:34.579038  2967 raft_consensus.cc:1275] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.580478  3047 consensus_queue.cc:1048] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf90e155681407ea8ab2905f6bd31b0" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.595503  2771 raft_consensus.cc:1275] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Refusing update from remote peer 0cb4525738614d1a8b56c5af1d7a7cae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.597783  3086 consensus_queue.cc:1048] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9352e7947904c43a959a977679d8864" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:34.688202 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:34.722029 24421 tablet_server.cc:179] TabletServer@127.23.217.65:34889 shutting down...
I20260709 12:22:34.755478 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:34.756345 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.757241 24421 raft_consensus.cc:2243] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.757735 24421 raft_consensus.cc:2272] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.761204 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.762100 24421 raft_consensus.cc:2243] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:34.762220  2736 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 86 similar messages]
I20260709 12:22:34.762877 24421 raft_consensus.cc:2272] T a737173da1a44edf98cb7483d880e113 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.766229 24421 tablet_replica.cc:333] stopping tablet replica
W20260709 12:22:34.766275  2736 consensus_peers.cc:597] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:34.766961 24421 raft_consensus.cc:2243] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.767508 24421 raft_consensus.cc:2272] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cf90e155681407ea8ab2905f6bd31b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.770128 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.770936 24421 raft_consensus.cc:2243] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.771374 24421 raft_consensus.cc:2272] T 3f0b5e339cec42f699083014deb0f51a P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.773782 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.774336 24421 raft_consensus.cc:2243] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.774869 24421 raft_consensus.cc:2272] T bdeb55b57c25429f9647fc81d00861da P 0cf90e155681407ea8ab2905f6bd31b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.778005 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.778733 24421 raft_consensus.cc:2243] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.779235 24421 raft_consensus.cc:2272] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.781584 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.782092 24421 raft_consensus.cc:2243] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.782612 24421 raft_consensus.cc:2272] T e42c4862903745279ec09f631252173d P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.784813 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.785355 24421 raft_consensus.cc:2243] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.785740 24421 raft_consensus.cc:2272] T 6fc5e66f25ff411090cd8bb264a2451f P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.788105 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.788621 24421 raft_consensus.cc:2243] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.789083 24421 raft_consensus.cc:2272] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cf90e155681407ea8ab2905f6bd31b0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.792244 24421 tablet_replica.cc:333] stopping tablet replica
I20260709 12:22:34.792780 24421 raft_consensus.cc:2243] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.793292 24421 raft_consensus.cc:2272] T 1507d1bea1424ff89b079b67055e8896 P 0cf90e155681407ea8ab2905f6bd31b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.828434 24421 tablet_server.cc:196] TabletServer@127.23.217.65:34889 shutdown complete.
W20260709 12:22:34.863210  2736 consensus_peers.cc:597] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:34.866144 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
W20260709 12:22:34.879644  2662 consensus_peers.cc:597] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:34.896376  2569 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:22:34.900614 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:34.901451 24421 tablet_replica.cc:333] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.902288 24421 raft_consensus.cc:2243] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.902818 24421 raft_consensus.cc:2272] T e42c4862903745279ec09f631252173d P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.905323 24421 tablet_replica.cc:333] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.905905 24421 raft_consensus.cc:2243] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.906335 24421 raft_consensus.cc:2272] T 4511dc9b60d046a49fa1c3a1234cdc29 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.908490 24421 tablet_replica.cc:333] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.908958 24421 raft_consensus.cc:2243] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:34.909689 24421 raft_consensus.cc:2272] T bdeb55b57c25429f9647fc81d00861da P 0cb4525738614d1a8b56c5af1d7a7cae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.912621 24421 tablet_replica.cc:333] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.913218 24421 raft_consensus.cc:2243] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:34.914040 24421 raft_consensus.cc:2272] T 8f72a8b042f84bf0b1691128bac6b0f4 P 0cb4525738614d1a8b56c5af1d7a7cae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.916427 24421 tablet_replica.cc:333] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.917171 24421 raft_consensus.cc:2243] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:34.918108 24421 raft_consensus.cc:2272] T a737173da1a44edf98cb7483d880e113 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.920339 24421 tablet_replica.cc:333] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.921195 24421 raft_consensus.cc:2243] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.921823 24421 raft_consensus.cc:2272] T 1507d1bea1424ff89b079b67055e8896 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.924851 24421 tablet_replica.cc:333] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.925518 24421 raft_consensus.cc:2243] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 LEADER]: Raft consensus shutting down.
W20260709 12:22:34.925812  2662 consensus_peers.cc:597] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:34.926882 24421 raft_consensus.cc:2272] T 21eeb3b649044104bf06aa1fcd15cb99 P 0cb4525738614d1a8b56c5af1d7a7cae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.929788 24421 tablet_replica.cc:333] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.930363 24421 raft_consensus.cc:2243] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:34.931354 24421 raft_consensus.cc:2272] T 6fc5e66f25ff411090cd8bb264a2451f P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.933472 24421 tablet_replica.cc:333] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.933959 24421 raft_consensus.cc:2243] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:34.934461 24421 raft_consensus.cc:2272] T 3f0b5e339cec42f699083014deb0f51a P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:34.936748 24421 tablet_replica.cc:333] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae: stopping tablet replica
I20260709 12:22:34.937254 24421 raft_consensus.cc:2243] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:34.938144 24421 raft_consensus.cc:2272] T 57ec28a1fdb64950a95df22fc2b70a98 P 0cb4525738614d1a8b56c5af1d7a7cae [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:34.977766  2736 consensus_peers.cc:597] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 -> Peer 0cf90e155681407ea8ab2905f6bd31b0 (127.23.217.65:34889): Couldn't send request to peer 0cf90e155681407ea8ab2905f6bd31b0. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:34.997928 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
I20260709 12:22:35.024610 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
W20260709 12:22:35.050328  2738 consensus_peers.cc:597] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 -> Peer 0cb4525738614d1a8b56c5af1d7a7cae (127.23.217.66:41357): Couldn't send request to peer 0cb4525738614d1a8b56c5af1d7a7cae. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:41357: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:35.062732 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:35.063774 24421 tablet_replica.cc:333] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.064620 24421 raft_consensus.cc:2243] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.065632 24421 raft_consensus.cc:2272] T e42c4862903745279ec09f631252173d P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.068257 24421 tablet_replica.cc:333] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.068902 24421 raft_consensus.cc:2243] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.069948 24421 raft_consensus.cc:2272] T 1507d1bea1424ff89b079b67055e8896 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.072662 24421 tablet_replica.cc:333] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.073341 24421 raft_consensus.cc:2243] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.073793 24421 raft_consensus.cc:2272] T 21eeb3b649044104bf06aa1fcd15cb99 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.076764 24421 tablet_replica.cc:333] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.077433 24421 raft_consensus.cc:2243] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.077901 24421 raft_consensus.cc:2272] T 6fc5e66f25ff411090cd8bb264a2451f P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.080210 24421 tablet_replica.cc:333] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.080785 24421 raft_consensus.cc:2243] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.081741 24421 raft_consensus.cc:2272] T 4511dc9b60d046a49fa1c3a1234cdc29 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.084193 24421 tablet_replica.cc:333] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.084751 24421 raft_consensus.cc:2243] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.085194 24421 raft_consensus.cc:2272] T bdeb55b57c25429f9647fc81d00861da P e9352e7947904c43a959a977679d8864 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.088124 24421 tablet_replica.cc:333] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.088702 24421 raft_consensus.cc:2243] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.089216 24421 raft_consensus.cc:2272] T 8f72a8b042f84bf0b1691128bac6b0f4 P e9352e7947904c43a959a977679d8864 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.091576 24421 tablet_replica.cc:333] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.092121 24421 raft_consensus.cc:2243] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.092646 24421 raft_consensus.cc:2272] T a737173da1a44edf98cb7483d880e113 P e9352e7947904c43a959a977679d8864 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.095295 24421 tablet_replica.cc:333] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.095855 24421 raft_consensus.cc:2243] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.096771 24421 raft_consensus.cc:2272] T 3f0b5e339cec42f699083014deb0f51a P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.099004 24421 tablet_replica.cc:333] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864: stopping tablet replica
I20260709 12:22:35.099493 24421 raft_consensus.cc:2243] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:35.099967 24421 raft_consensus.cc:2272] T 57ec28a1fdb64950a95df22fc2b70a98 P e9352e7947904c43a959a977679d8864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.137347 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
I20260709 12:22:35.165292 24421 master.cc:562] Master@127.23.217.126:42545 shutting down...
I20260709 12:22:35.191728 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:35.192272 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:35.192570 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P 83a0e1d1a7d54d5290dba0bb1b180006: stopping tablet replica
I20260709 12:22:35.232255 24421 master.cc:584] Master@127.23.217.126:42545 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13551 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:22:35.282048 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:34843
I20260709 12:22:35.283200 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:35.289163  3095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:35.289388  3096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:35.290365  3098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:35.290933 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:35.291934 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:35.292117 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:35.292299 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599755292284 us; error 0 us; skew 500 ppm
I20260709 12:22:35.292873 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:35.295465 24421 webserver.cc:533] Webserver started at http://127.23.217.126:41155/ using document root <none> and password file <none>
I20260709 12:22:35.295970 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:35.296135 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:35.296449 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:35.297560 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "b75269998545475293d889b252b539d5"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.302333 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:22:35.306362  3103 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.307415 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:35.307761 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "b75269998545475293d889b252b539d5"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.308095 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:35.319299 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:35.320547 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:35.363945 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:34843
I20260709 12:22:35.364113  3154 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:34843 every 8 connection(s)
I20260709 12:22:35.368513  3155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:35.382170  3155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5: Bootstrap starting.
I20260709 12:22:35.387918  3155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.393033  3155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5: No bootstrap required, opened a new log
I20260709 12:22:35.395694  3155 raft_consensus.cc:359] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b75269998545475293d889b252b539d5" member_type: VOTER }
I20260709 12:22:35.396126  3155 raft_consensus.cc:385] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.396425  3155 raft_consensus.cc:740] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75269998545475293d889b252b539d5, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.397148  3155 consensus_queue.cc:260] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b75269998545475293d889b252b539d5" member_type: VOTER }
I20260709 12:22:35.397660  3155 raft_consensus.cc:399] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:35.397867  3155 raft_consensus.cc:493] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:35.398145  3155 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:35.404273  3155 raft_consensus.cc:515] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b75269998545475293d889b252b539d5" member_type: VOTER }
I20260709 12:22:35.404882  3155 leader_election.cc:304] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b75269998545475293d889b252b539d5; no voters: 
I20260709 12:22:35.406206  3155 leader_election.cc:290] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:35.406661  3158 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:35.408471  3158 raft_consensus.cc:697] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 1 LEADER]: Becoming Leader. State: Replica: b75269998545475293d889b252b539d5, State: Running, Role: LEADER
I20260709 12:22:35.409180  3158 consensus_queue.cc:237] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b75269998545475293d889b252b539d5" member_type: VOTER }
I20260709 12:22:35.409644  3155 sys_catalog.cc:565] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:35.414067  3160 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b75269998545475293d889b252b539d5. Latest consensus state: current_term: 1 leader_uuid: "b75269998545475293d889b252b539d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b75269998545475293d889b252b539d5" member_type: VOTER } }
I20260709 12:22:35.414808  3160 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:35.416285  3159 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b75269998545475293d889b252b539d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b75269998545475293d889b252b539d5" member_type: VOTER } }
I20260709 12:22:35.417055  3159 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:35.423077  3166 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:35.427812  3166 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:35.429148 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:35.437516  3166 catalog_manager.cc:1383] Generated new cluster ID: accd5a7793a84a379179b745b9a22f6e
I20260709 12:22:35.437892  3166 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:35.461517  3166 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:35.463197  3166 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:35.476504  3166 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5: Generated new TSK 0
I20260709 12:22:35.477246  3166 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:35.495978 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:35.502583  3176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:35.503713  3177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:35.505721  3179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:35.506228 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:35.507324 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:35.507514 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:35.507733 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599755507698 us; error 0 us; skew 500 ppm
I20260709 12:22:35.508307 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:35.511166 24421 webserver.cc:533] Webserver started at http://127.23.217.65:36983/ using document root <none> and password file <none>
I20260709 12:22:35.511714 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:35.511974 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:35.512254 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:35.513345 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "af2b734505a4432e93c11f5ae9baa0db"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.518143 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 12:22:35.521796  3184 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.522904 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:35.523334 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "af2b734505a4432e93c11f5ae9baa0db"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.523615 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:35.537266 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:35.538582 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:35.540024 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:35.542331 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:35.542578 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:22:35.542863 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:35.543058 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.589017 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:45689
I20260709 12:22:35.589178  3246 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:45689 every 8 connection(s)
I20260709 12:22:35.594213 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:35.601706  3251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:35.603654  3252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:35.607527 24421 server_base.cc:1061] running on GCE node
W20260709 12:22:35.607406  3254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:35.609562  3247 heartbeater.cc:344] Connected to a master server at 127.23.217.126:34843
I20260709 12:22:35.609889 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:22:35.610031  3247 heartbeater.cc:461] Registering TS with master...
W20260709 12:22:35.610198 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:35.610530 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599755610481 us; error 0 us; skew 500 ppm
I20260709 12:22:35.611199 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:35.611159  3247 heartbeater.cc:507] Master 127.23.217.126:34843 requested a full tablet report, sending...
I20260709 12:22:35.614023  3120 ts_manager.cc:194] Registered new tserver with Master: af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689)
I20260709 12:22:35.614475 24421 webserver.cc:533] Webserver started at http://127.23.217.66:40515/ using document root <none> and password file <none>
I20260709 12:22:35.615092 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:35.615288 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:35.615577 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:35.616369  3120 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41092
I20260709 12:22:35.616796 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "db0bf45c4990494892675b3d9ff0bb5d"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.621687 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:22:35.625295  3259 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.626191 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:22:35.626547 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "db0bf45c4990494892675b3d9ff0bb5d"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.626832 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:35.639855 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:35.641042 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:35.642640 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:35.644910 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:35.645136 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.645386 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:35.645555 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.690615 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:41421
I20260709 12:22:35.690765  3321 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:41421 every 8 connection(s)
I20260709 12:22:35.695617 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:35.704423  3326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:35.705195  3325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:35.708194  3328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:35.710218  3322 heartbeater.cc:344] Connected to a master server at 127.23.217.126:34843
I20260709 12:22:35.710759  3322 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:35.710949 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:35.711891  3322 heartbeater.cc:507] Master 127.23.217.126:34843 requested a full tablet report, sending...
I20260709 12:22:35.712167 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:35.712483 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:35.712711 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599755712690 us; error 0 us; skew 500 ppm
I20260709 12:22:35.713434 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:35.714171  3120 ts_manager.cc:194] Registered new tserver with Master: db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:35.715869  3120 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41094
I20260709 12:22:35.716395 24421 webserver.cc:533] Webserver started at http://127.23.217.67:43961/ using document root <none> and password file <none>
I20260709 12:22:35.717190 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:35.717448 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:35.717743 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:35.719122 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.724064 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:35.727752  3333 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.728724 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:35.729054 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
format_stamp: "Formatted at 2026-07-09 12:22:35 on dist-test-slave-lg8f"
I20260709 12:22:35.729363 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:35.745981 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:35.747370 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:35.749197 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:35.751955 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:35.752157 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.752346 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:35.752482 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:35.799283 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:34753
I20260709 12:22:35.799404  3395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:34753 every 8 connection(s)
I20260709 12:22:35.814718  3396 heartbeater.cc:344] Connected to a master server at 127.23.217.126:34843
I20260709 12:22:35.815172  3396 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:35.815951  3396 heartbeater.cc:507] Master 127.23.217.126:34843 requested a full tablet report, sending...
I20260709 12:22:35.818011  3120 ts_manager.cc:194] Registered new tserver with Master: d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:35.818199 24421 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015319136s
I20260709 12:22:35.819931  3120 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41100
I20260709 12:22:35.848559  3120 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41106:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:35.850693  3120 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:35.908849  3212 tablet_service.cc:1511] Processing CreateTablet for tablet 8a085a1f703441c2adb789412b0e5319 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e1b0fdf795994a0f89ffac8971fcfadc]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:35.909096  3211 tablet_service.cc:1511] Processing CreateTablet for tablet 7a3ec06650d54d67a3d32b8ead1d136e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e1b0fdf795994a0f89ffac8971fcfadc]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:35.910727  3211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a3ec06650d54d67a3d32b8ead1d136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:35.915086  3212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a085a1f703441c2adb789412b0e5319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:35.916414  3287 tablet_service.cc:1511] Processing CreateTablet for tablet 8a085a1f703441c2adb789412b0e5319 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e1b0fdf795994a0f89ffac8971fcfadc]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:35.916903  3286 tablet_service.cc:1511] Processing CreateTablet for tablet 7a3ec06650d54d67a3d32b8ead1d136e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e1b0fdf795994a0f89ffac8971fcfadc]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:35.917858  3287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a085a1f703441c2adb789412b0e5319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:35.918848  3286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a3ec06650d54d67a3d32b8ead1d136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:35.926458  3361 tablet_service.cc:1511] Processing CreateTablet for tablet 8a085a1f703441c2adb789412b0e5319 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e1b0fdf795994a0f89ffac8971fcfadc]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:35.927927  3361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a085a1f703441c2adb789412b0e5319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:35.938634  3360 tablet_service.cc:1511] Processing CreateTablet for tablet 7a3ec06650d54d67a3d32b8ead1d136e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e1b0fdf795994a0f89ffac8971fcfadc]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:35.940083  3360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a3ec06650d54d67a3d32b8ead1d136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:35.955396  3411 tablet_bootstrap.cc:492] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db: Bootstrap starting.
I20260709 12:22:35.962323  3412 tablet_bootstrap.cc:492] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d: Bootstrap starting.
I20260709 12:22:35.964308  3413 tablet_bootstrap.cc:492] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10: Bootstrap starting.
I20260709 12:22:35.970468  3412 tablet_bootstrap.cc:654] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.971150  3413 tablet_bootstrap.cc:654] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.971823  3411 tablet_bootstrap.cc:654] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:35.976605  3412 tablet_bootstrap.cc:492] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d: No bootstrap required, opened a new log
I20260709 12:22:35.977106  3412 ts_tablet_manager.cc:1403] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:22:35.977761  3413 tablet_bootstrap.cc:492] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10: No bootstrap required, opened a new log
I20260709 12:22:35.978076  3411 tablet_bootstrap.cc:492] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db: No bootstrap required, opened a new log
I20260709 12:22:35.978345  3413 ts_tablet_manager.cc:1403] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent bootstrapping tablet: real 0.014s	user 0.003s	sys 0.008s
I20260709 12:22:35.978585  3411 ts_tablet_manager.cc:1403] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.000s
I20260709 12:22:35.980020  3412 raft_consensus.cc:359] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:35.980772  3412 raft_consensus.cc:385] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.981072  3412 raft_consensus.cc:740] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0bf45c4990494892675b3d9ff0bb5d, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.982053  3411 raft_consensus.cc:359] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:35.981897  3413 raft_consensus.cc:359] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:35.982189  3412 consensus_queue.cc:260] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:35.982890  3411 raft_consensus.cc:385] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.982926  3413 raft_consensus.cc:385] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:35.983424  3411 raft_consensus.cc:740] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.983464  3413 raft_consensus.cc:740] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Initialized, Role: FOLLOWER
I20260709 12:22:35.984371  3413 consensus_queue.cc:260] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:35.984586  3411 consensus_queue.cc:260] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:35.990805  3322 heartbeater.cc:499] Master 127.23.217.126:34843 was elected leader, sending a full tablet report...
I20260709 12:22:35.991972  3412 ts_tablet_manager.cc:1434] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.004s
I20260709 12:22:35.993196  3412 tablet_bootstrap.cc:492] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d: Bootstrap starting.
I20260709 12:22:36.000515  3412 tablet_bootstrap.cc:654] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.004668  3247 heartbeater.cc:499] Master 127.23.217.126:34843 was elected leader, sending a full tablet report...
I20260709 12:22:36.004807  3413 ts_tablet_manager.cc:1434] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent starting tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 12:22:36.005120  3411 ts_tablet_manager.cc:1434] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.019s
I20260709 12:22:36.006254  3411 tablet_bootstrap.cc:492] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db: Bootstrap starting.
I20260709 12:22:36.007395  3396 heartbeater.cc:499] Master 127.23.217.126:34843 was elected leader, sending a full tablet report...
I20260709 12:22:36.014043  3413 tablet_bootstrap.cc:492] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10: Bootstrap starting.
I20260709 12:22:36.014391  3411 tablet_bootstrap.cc:654] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.017094  3412 tablet_bootstrap.cc:492] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d: No bootstrap required, opened a new log
I20260709 12:22:36.017848  3412 ts_tablet_manager.cc:1403] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260709 12:22:36.021512  3412 raft_consensus.cc:359] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.022574  3412 raft_consensus.cc:385] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.022015  3413 tablet_bootstrap.cc:654] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.023056  3412 raft_consensus.cc:740] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0bf45c4990494892675b3d9ff0bb5d, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.024081  3412 consensus_queue.cc:260] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.026459  3412 ts_tablet_manager.cc:1434] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:36.036386  3413 tablet_bootstrap.cc:492] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10: No bootstrap required, opened a new log
I20260709 12:22:36.037215  3413 ts_tablet_manager.cc:1403] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.006s
I20260709 12:22:36.039889  3413 raft_consensus.cc:359] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.040305  3411 tablet_bootstrap.cc:492] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db: No bootstrap required, opened a new log
I20260709 12:22:36.040664  3413 raft_consensus.cc:385] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.041031  3411 ts_tablet_manager.cc:1403] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.015s
I20260709 12:22:36.041040  3413 raft_consensus.cc:740] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.041975  3413 consensus_queue.cc:260] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.043691  3413 ts_tablet_manager.cc:1434] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 12:22:36.044368  3411 raft_consensus.cc:359] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.045141  3411 raft_consensus.cc:385] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.045456  3411 raft_consensus.cc:740] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.046203  3411 consensus_queue.cc:260] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.048338  3411 ts_tablet_manager.cc:1434] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:36.105242  3419 raft_consensus.cc:493] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.105902  3419 raft_consensus.cc:515] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.109375  3419 leader_election.cc:290] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:36.120783  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d" is_pre_election: true
I20260709 12:22:36.121448  3371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" is_pre_election: true
I20260709 12:22:36.121721  3297 raft_consensus.cc:2468] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 0.
I20260709 12:22:36.122242  3371 raft_consensus.cc:2468] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 0.
I20260709 12:22:36.123210  3187 leader_election.cc:304] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, db0bf45c4990494892675b3d9ff0bb5d; no voters: 
I20260709 12:22:36.124089  3419 raft_consensus.cc:2804] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:36.124578  3419 raft_consensus.cc:493] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.124902  3419 raft_consensus.cc:3060] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.131791  3419 raft_consensus.cc:515] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.133805  3419 leader_election.cc:290] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 election: Requested vote from peers db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:36.134781  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.135128  3371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
I20260709 12:22:36.135430  3297 raft_consensus.cc:3060] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.135921  3371 raft_consensus.cc:3060] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.142123  3371 raft_consensus.cc:2468] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 1.
I20260709 12:22:36.142223  3297 raft_consensus.cc:2468] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 1.
I20260709 12:22:36.143528  3186 leader_election.cc:304] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, d6e3cf60df4d456fadd8de78cf7b6f10; no voters: 
I20260709 12:22:36.144328  3419 raft_consensus.cc:2804] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:36.145339  3419 raft_consensus.cc:697] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 LEADER]: Becoming Leader. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Running, Role: LEADER
I20260709 12:22:36.146143  3419 consensus_queue.cc:237] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.153712  3120 catalog_manager.cc:5697] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db reported cstate change: term changed from 0 to 1, leader changed from <none> to af2b734505a4432e93c11f5ae9baa0db (127.23.217.65). New cstate: current_term: 1 leader_uuid: "af2b734505a4432e93c11f5ae9baa0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:36.211195  3418 raft_consensus.cc:493] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.211836  3418 raft_consensus.cc:515] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.213713  3419 raft_consensus.cc:493] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.214418  3419 raft_consensus.cc:515] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.215561  3418 leader_election.cc:290] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:36.218714  3419 leader_election.cc:290] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:36.218840  3371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" is_pre_election: true
I20260709 12:22:36.219592  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d" is_pre_election: true
I20260709 12:22:36.219696  3371 raft_consensus.cc:2468] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 0.
I20260709 12:22:36.220304  3297 raft_consensus.cc:2468] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 0.
I20260709 12:22:36.221114  3186 leader_election.cc:304] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, d6e3cf60df4d456fadd8de78cf7b6f10; no voters: 
I20260709 12:22:36.222072  3419 raft_consensus.cc:2804] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:36.222481  3419 raft_consensus.cc:493] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.222925  3419 raft_consensus.cc:3060] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.227479  3222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2b734505a4432e93c11f5ae9baa0db" is_pre_election: true
I20260709 12:22:36.229039  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d" is_pre_election: true
I20260709 12:22:36.229744  3297 raft_consensus.cc:2468] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 0.
I20260709 12:22:36.230672  3419 raft_consensus.cc:515] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.231038  3336 leader_election.cc:304] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6e3cf60df4d456fadd8de78cf7b6f10, db0bf45c4990494892675b3d9ff0bb5d; no voters: 
I20260709 12:22:36.231980  3418 raft_consensus.cc:2804] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:36.232496  3418 raft_consensus.cc:493] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.232838  3418 raft_consensus.cc:3060] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.233513  3419 leader_election.cc:290] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 election: Requested vote from peers db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:36.233795  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.234373  3297 raft_consensus.cc:3060] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.234423  3371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
I20260709 12:22:36.240007  3297 raft_consensus.cc:2468] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 1.
I20260709 12:22:36.239784  3418 raft_consensus.cc:515] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.232120  3222 raft_consensus.cc:2393] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d6e3cf60df4d456fadd8de78cf7b6f10 in current term 1: Already voted for candidate af2b734505a4432e93c11f5ae9baa0db in this term.
I20260709 12:22:36.241219  3187 leader_election.cc:304] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, db0bf45c4990494892675b3d9ff0bb5d; no voters: 
I20260709 12:22:36.241600  3371 raft_consensus.cc:2393] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate af2b734505a4432e93c11f5ae9baa0db in current term 1: Already voted for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in this term.
I20260709 12:22:36.242539  3419 raft_consensus.cc:2804] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:36.243134  3419 raft_consensus.cc:697] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 1 LEADER]: Becoming Leader. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Running, Role: LEADER
I20260709 12:22:36.243673  3418 leader_election.cc:290] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 election: Requested vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:36.243703  3221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
I20260709 12:22:36.244822  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a085a1f703441c2adb789412b0e5319" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.244887  3419 consensus_queue.cc:237] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.246389  3297 raft_consensus.cc:2393] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d6e3cf60df4d456fadd8de78cf7b6f10 in current term 1: Already voted for candidate af2b734505a4432e93c11f5ae9baa0db in this term.
I20260709 12:22:36.247501  3336 leader_election.cc:304] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6e3cf60df4d456fadd8de78cf7b6f10; no voters: af2b734505a4432e93c11f5ae9baa0db, db0bf45c4990494892675b3d9ff0bb5d
I20260709 12:22:36.248308  3418 raft_consensus.cc:2749] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:36.253072  3120 catalog_manager.cc:5697] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db reported cstate change: term changed from 0 to 1, leader changed from <none> to af2b734505a4432e93c11f5ae9baa0db (127.23.217.65). New cstate: current_term: 1 leader_uuid: "af2b734505a4432e93c11f5ae9baa0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:36.319500  3120 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41108:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:36.321869  3120 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:36.347157  3211 tablet_service.cc:1511] Processing CreateTablet for tablet 55d0ad633342440f85cd87b29fc4eb1f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=bf2adcef741149a9a19c94b8958ac809]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:36.348608  3211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d0ad633342440f85cd87b29fc4eb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:36.348333  3287 tablet_service.cc:1511] Processing CreateTablet for tablet 55d0ad633342440f85cd87b29fc4eb1f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=bf2adcef741149a9a19c94b8958ac809]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:36.349903  3287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d0ad633342440f85cd87b29fc4eb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:36.350136  3360 tablet_service.cc:1511] Processing CreateTablet for tablet 55d0ad633342440f85cd87b29fc4eb1f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=bf2adcef741149a9a19c94b8958ac809]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:22:36.351462  3360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d0ad633342440f85cd87b29fc4eb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:36.353942  3212 tablet_service.cc:1511] Processing CreateTablet for tablet c1905c80d100456581b937ce60c970d1 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=bf2adcef741149a9a19c94b8958ac809]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:36.355337  3212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1905c80d100456581b937ce60c970d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:36.355532  3286 tablet_service.cc:1511] Processing CreateTablet for tablet c1905c80d100456581b937ce60c970d1 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=bf2adcef741149a9a19c94b8958ac809]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:36.356832  3286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1905c80d100456581b937ce60c970d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:36.359886  3361 tablet_service.cc:1511] Processing CreateTablet for tablet c1905c80d100456581b937ce60c970d1 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=bf2adcef741149a9a19c94b8958ac809]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:22:36.361258  3361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1905c80d100456581b937ce60c970d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:36.375299  3412 tablet_bootstrap.cc:492] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d: Bootstrap starting.
I20260709 12:22:36.383478  3412 tablet_bootstrap.cc:654] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.385074  3413 tablet_bootstrap.cc:492] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10: Bootstrap starting.
I20260709 12:22:36.387131  3411 tablet_bootstrap.cc:492] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db: Bootstrap starting.
I20260709 12:22:36.392493  3413 tablet_bootstrap.cc:654] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.394081  3411 tablet_bootstrap.cc:654] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.398674  3413 tablet_bootstrap.cc:492] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10: No bootstrap required, opened a new log
I20260709 12:22:36.398691  3412 tablet_bootstrap.cc:492] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d: No bootstrap required, opened a new log
I20260709 12:22:36.399391  3413 ts_tablet_manager.cc:1403] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 12:22:36.399412  3412 ts_tablet_manager.cc:1403] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.004s
I20260709 12:22:36.400359  3411 tablet_bootstrap.cc:492] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db: No bootstrap required, opened a new log
I20260709 12:22:36.400923  3411 ts_tablet_manager.cc:1403] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 12:22:36.402936  3413 raft_consensus.cc:359] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.403189  3412 raft_consensus.cc:359] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.403821  3413 raft_consensus.cc:385] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.403896  3412 raft_consensus.cc:385] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.404263  3413 raft_consensus.cc:740] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.404333  3412 raft_consensus.cc:740] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0bf45c4990494892675b3d9ff0bb5d, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.404405  3411 raft_consensus.cc:359] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.405401  3411 raft_consensus.cc:385] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.406160  3411 raft_consensus.cc:740] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.405288  3413 consensus_queue.cc:260] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.405288  3412 consensus_queue.cc:260] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.407095  3411 consensus_queue.cc:260] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.408533  3413 ts_tablet_manager.cc:1434] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:22:36.409466  3411 ts_tablet_manager.cc:1434] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:36.409654  3413 tablet_bootstrap.cc:492] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10: Bootstrap starting.
I20260709 12:22:36.410822  3411 tablet_bootstrap.cc:492] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db: Bootstrap starting.
I20260709 12:22:36.418192  3413 tablet_bootstrap.cc:654] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:36.422901  3168 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:36.423095  3411 tablet_bootstrap.cc:654] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.426002  3412 ts_tablet_manager.cc:1434] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:22:36.426426  3418 raft_consensus.cc:493] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.427130  3418 raft_consensus.cc:515] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.429376  3418 leader_election.cc:290] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:36.431036  3412 tablet_bootstrap.cc:492] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: Bootstrap starting.
I20260709 12:22:36.433547  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d" is_pre_election: true
I20260709 12:22:36.434602  3297 raft_consensus.cc:2468] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 0.
I20260709 12:22:36.442044  3221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2b734505a4432e93c11f5ae9baa0db" is_pre_election: true
W20260709 12:22:36.444162  3334 leader_election.cc:343] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:36.444557  3412 tablet_bootstrap.cc:654] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:36.445158  3336 leader_election.cc:304] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6e3cf60df4d456fadd8de78cf7b6f10, db0bf45c4990494892675b3d9ff0bb5d; no voters: af2b734505a4432e93c11f5ae9baa0db
I20260709 12:22:36.445785  3413 tablet_bootstrap.cc:492] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10: No bootstrap required, opened a new log
I20260709 12:22:36.446439  3413 ts_tablet_manager.cc:1403] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.019s
I20260709 12:22:36.446619  3418 raft_consensus.cc:2804] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:36.447105  3418 raft_consensus.cc:493] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.447497  3418 raft_consensus.cc:3060] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.449764  3413 raft_consensus.cc:359] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.450418  3413 raft_consensus.cc:385] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.450739  3413 raft_consensus.cc:740] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.451511  3412 tablet_bootstrap.cc:492] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: No bootstrap required, opened a new log
I20260709 12:22:36.451484  3413 consensus_queue.cc:260] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.452142  3412 ts_tablet_manager.cc:1403] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 12:22:36.454636  3413 ts_tablet_manager.cc:1434] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:36.455554  3412 raft_consensus.cc:359] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.456374  3412 raft_consensus.cc:385] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.456687  3412 raft_consensus.cc:740] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0bf45c4990494892675b3d9ff0bb5d, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.457002  3418 raft_consensus.cc:515] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.457664  3412 consensus_queue.cc:260] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.460192  3412 ts_tablet_manager.cc:1434] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:36.460186  3221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
W20260709 12:22:36.461889  3334 leader_election.cc:343] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:36.462749  3418 leader_election.cc:290] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 election: Requested vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:36.463160  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.463841  3297 raft_consensus.cc:3060] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.465374  3411 tablet_bootstrap.cc:492] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db: No bootstrap required, opened a new log
I20260709 12:22:36.465909  3411 ts_tablet_manager.cc:1403] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.014s
I20260709 12:22:36.469118  3411 raft_consensus.cc:359] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.469833  3411 raft_consensus.cc:385] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:36.470139  3411 raft_consensus.cc:740] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Initialized, Role: FOLLOWER
I20260709 12:22:36.470882  3411 consensus_queue.cc:260] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.473193  3411 ts_tablet_manager.cc:1434] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:22:36.474185  3297 raft_consensus.cc:2468] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 1.
I20260709 12:22:36.475462  3336 leader_election.cc:304] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6e3cf60df4d456fadd8de78cf7b6f10, db0bf45c4990494892675b3d9ff0bb5d; no voters: af2b734505a4432e93c11f5ae9baa0db
I20260709 12:22:36.476272  3418 raft_consensus.cc:2804] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:36.476819  3418 raft_consensus.cc:697] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Becoming Leader. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Running, Role: LEADER
I20260709 12:22:36.477797  3418 consensus_queue.cc:237] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.483335  3418 raft_consensus.cc:493] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.483865  3418 raft_consensus.cc:515] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.486225  3418 leader_election.cc:290] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:36.487066  3221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2b734505a4432e93c11f5ae9baa0db" is_pre_election: true
I20260709 12:22:36.488018  3221 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 0.
I20260709 12:22:36.487951  3117 catalog_manager.cc:5697] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:36.489634  3334 leader_election.cc:304] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, d6e3cf60df4d456fadd8de78cf7b6f10; no voters: 
I20260709 12:22:36.489907  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d" is_pre_election: true
I20260709 12:22:36.490722  3297 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 0.
I20260709 12:22:36.490720  3418 raft_consensus.cc:2804] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:36.491794  3418 raft_consensus.cc:493] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.492173  3418 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.499657  3418 raft_consensus.cc:515] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.501492  3418 leader_election.cc:290] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 election: Requested vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:36.502296  3221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
I20260709 12:22:36.502744  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.502993  3221 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.503443  3297 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.508785  3221 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 1.
I20260709 12:22:36.509449  3297 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 1.
I20260709 12:22:36.510138  3334 leader_election.cc:304] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, d6e3cf60df4d456fadd8de78cf7b6f10; no voters: 
I20260709 12:22:36.511273  3418 raft_consensus.cc:2804] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:36.511772  3418 raft_consensus.cc:697] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Becoming Leader. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Running, Role: LEADER
I20260709 12:22:36.512565  3418 consensus_queue.cc:237] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.520619  3117 catalog_manager.cc:5697] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:36.567077  3221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e"
dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
mode: GRACEFUL
new_leader_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
 from {username='slave'} at 127.0.0.1:45968
I20260709 12:22:36.567711  3221 raft_consensus.cc:606] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 LEADER]: Received request to transfer leadership to TS d6e3cf60df4d456fadd8de78cf7b6f10
I20260709 12:22:36.581633  3371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f"
dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
mode: GRACEFUL
new_leader_uuid: "af2b734505a4432e93c11f5ae9baa0db"
 from {username='slave'} at 127.0.0.1:36324
I20260709 12:22:36.582114  3371 raft_consensus.cc:606] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Received request to transfer leadership to TS af2b734505a4432e93c11f5ae9baa0db
I20260709 12:22:36.587555  3371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1905c80d100456581b937ce60c970d1"
dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
mode: GRACEFUL
new_leader_uuid: "af2b734505a4432e93c11f5ae9baa0db"
 from {username='slave'} at 127.0.0.1:36324
I20260709 12:22:36.588058  3371 raft_consensus.cc:606] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Received request to transfer leadership to TS af2b734505a4432e93c11f5ae9baa0db
I20260709 12:22:36.661096  3419 consensus_queue.cc:1048] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:36.668558  3425 raft_consensus.cc:993] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db: : Instructing follower d6e3cf60df4d456fadd8de78cf7b6f10 to start an election
I20260709 12:22:36.669299  3419 raft_consensus.cc:1081] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 LEADER]: Signalling peer d6e3cf60df4d456fadd8de78cf7b6f10 to start an election
I20260709 12:22:36.671589  3370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e"
dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
 from {username='slave'} at 127.0.0.1:36310
I20260709 12:22:36.675100  3370 raft_consensus.cc:493] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:36.675698  3370 raft_consensus.cc:3060] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.679322  3445 consensus_queue.cc:1048] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:36.685271  3370 raft_consensus.cc:515] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.691300  3370 leader_election.cc:290] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 2 election: Requested vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421)
I20260709 12:22:36.692070  3221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
I20260709 12:22:36.692648  3296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a3ec06650d54d67a3d32b8ead1d136e" candidate_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.692893  3221 raft_consensus.cc:3055] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:36.693399  3221 raft_consensus.cc:740] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Running, Role: LEADER
I20260709 12:22:36.694339  3221 consensus_queue.cc:260] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.695691  3221 raft_consensus.cc:3060] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.703438  3221 raft_consensus.cc:2468] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in term 2.
I20260709 12:22:36.704329  3445 raft_consensus.cc:493] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.705077  3334 leader_election.cc:304] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, d6e3cf60df4d456fadd8de78cf7b6f10; no voters: db0bf45c4990494892675b3d9ff0bb5d
I20260709 12:22:36.704891  3445 raft_consensus.cc:515] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.706280  3418 raft_consensus.cc:2804] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:36.706805  3418 raft_consensus.cc:697] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 LEADER]: Becoming Leader. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Running, Role: LEADER
I20260709 12:22:36.707412  3445 leader_election.cc:290] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:36.707664  3418 consensus_queue.cc:237] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.709090  3370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" is_pre_election: true
I20260709 12:22:36.709224  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d" is_pre_election: true
I20260709 12:22:36.711565  3297 raft_consensus.cc:2393] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af2b734505a4432e93c11f5ae9baa0db in current term 1: Already voted for candidate d6e3cf60df4d456fadd8de78cf7b6f10 in this term.
I20260709 12:22:36.712754  3187 leader_election.cc:304] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db; no voters: d6e3cf60df4d456fadd8de78cf7b6f10, db0bf45c4990494892675b3d9ff0bb5d
I20260709 12:22:36.713658  3445 raft_consensus.cc:3060] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:36.718030  3116 catalog_manager.cc:5697] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 reported cstate change: term changed from 1 to 2, leader changed from af2b734505a4432e93c11f5ae9baa0db (127.23.217.65) to d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:36.723167  3445 raft_consensus.cc:2749] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:36.802420  3445 consensus_queue.cc:1048] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:36.815953  3425 consensus_queue.cc:1048] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:36.860987  3418 consensus_queue.cc:1048] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:36.868065  3440 raft_consensus.cc:993] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10: : Instructing follower af2b734505a4432e93c11f5ae9baa0db to start an election
I20260709 12:22:36.868829  3418 raft_consensus.cc:1081] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Signalling peer af2b734505a4432e93c11f5ae9baa0db to start an election
I20260709 12:22:36.872516  3222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1905c80d100456581b937ce60c970d1"
dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
 from {username='slave'} at 127.0.0.1:45956
I20260709 12:22:36.873126  3222 raft_consensus.cc:493] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:36.873530  3222 raft_consensus.cc:3060] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.874990  3418 consensus_queue.cc:1048] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:36.883944  3222 raft_consensus.cc:515] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.886415  3222 leader_election.cc:290] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 2 election: Requested vote from peers db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:36.887316  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.887516  3370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1905c80d100456581b937ce60c970d1" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
I20260709 12:22:36.888228  3297 raft_consensus.cc:3060] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.888228  3370 raft_consensus.cc:3055] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:36.888888  3370 raft_consensus.cc:740] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Running, Role: LEADER
I20260709 12:22:36.889760  3370 consensus_queue.cc:260] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.891104  3370 raft_consensus.cc:3060] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.896600  3297 raft_consensus.cc:2468] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 2.
I20260709 12:22:36.898208  3370 raft_consensus.cc:2468] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 2.
I20260709 12:22:36.898223  3187 leader_election.cc:304] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, db0bf45c4990494892675b3d9ff0bb5d; no voters: 
I20260709 12:22:36.899433  3419 raft_consensus.cc:2804] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:36.900102  3419 raft_consensus.cc:697] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 2 LEADER]: Becoming Leader. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Running, Role: LEADER
I20260709 12:22:36.901129  3419 consensus_queue.cc:237] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.909080  3418 consensus_queue.cc:1048] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:36.909142  3119 catalog_manager.cc:5697] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db reported cstate change: term changed from 1 to 2, leader changed from d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67) to af2b734505a4432e93c11f5ae9baa0db (127.23.217.65). New cstate: current_term: 2 leader_uuid: "af2b734505a4432e93c11f5ae9baa0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:36.926450  3440 consensus_queue.cc:1048] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:36.934885  3418 raft_consensus.cc:993] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10: : Instructing follower af2b734505a4432e93c11f5ae9baa0db to start an election
I20260709 12:22:36.935433  3440 raft_consensus.cc:1081] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Signalling peer af2b734505a4432e93c11f5ae9baa0db to start an election
I20260709 12:22:36.937362  3222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f"
dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
 from {username='slave'} at 127.0.0.1:45956
I20260709 12:22:36.938015  3222 raft_consensus.cc:493] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:36.938333  3222 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.943673  3222 raft_consensus.cc:515] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.945758  3222 leader_election.cc:290] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 2 election: Requested vote from peers db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66:41421), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:36.946758  3297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
I20260709 12:22:36.947528  3370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "af2b734505a4432e93c11f5ae9baa0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
I20260709 12:22:36.947587  3297 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.948180  3370 raft_consensus.cc:3055] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:36.948520  3370 raft_consensus.cc:740] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d6e3cf60df4d456fadd8de78cf7b6f10, State: Running, Role: LEADER
I20260709 12:22:36.949252  3370 consensus_queue.cc:260] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.950596  3370 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.955196  3297 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 2.
I20260709 12:22:36.956511  3187 leader_election.cc:304] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2b734505a4432e93c11f5ae9baa0db, db0bf45c4990494892675b3d9ff0bb5d; no voters: 
I20260709 12:22:36.956583  3370 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2b734505a4432e93c11f5ae9baa0db in term 2.
I20260709 12:22:36.957383  3419 raft_consensus.cc:2804] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:36.957890  3419 raft_consensus.cc:697] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 LEADER]: Becoming Leader. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Running, Role: LEADER
I20260709 12:22:36.958683  3419 consensus_queue.cc:237] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:36.964978  3119 catalog_manager.cc:5697] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db reported cstate change: term changed from 1 to 2, leader changed from d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67) to af2b734505a4432e93c11f5ae9baa0db (127.23.217.65). New cstate: current_term: 2 leader_uuid: "af2b734505a4432e93c11f5ae9baa0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:37.084170 24421 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:22:37.085836 24421 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:22:37.086138 24421 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:22:37.098135  3222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f"
dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
mode: GRACEFUL
new_leader_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:22:37.098855  3222 raft_consensus.cc:606] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 LEADER]: Received request to transfer leadership to TS db0bf45c4990494892675b3d9ff0bb5d
I20260709 12:22:37.100023 24421 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:22:37.312855  3297 raft_consensus.cc:3060] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:37.317715  3222 raft_consensus.cc:1275] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Refusing update from remote peer d6e3cf60df4d456fadd8de78cf7b6f10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:37.319669  3440 consensus_queue.cc:1048] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:37.322403  3297 raft_consensus.cc:1275] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Refusing update from remote peer d6e3cf60df4d456fadd8de78cf7b6f10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:37.323838  3440 consensus_queue.cc:1048] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:37.371191  3297 raft_consensus.cc:1275] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Refusing update from remote peer af2b734505a4432e93c11f5ae9baa0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:37.372788  3419 consensus_queue.cc:1048] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:37.396756  3370 raft_consensus.cc:1275] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Refusing update from remote peer af2b734505a4432e93c11f5ae9baa0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:37.398880  3462 consensus_queue.cc:1048] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:37.435998  3370 raft_consensus.cc:1275] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Refusing update from remote peer af2b734505a4432e93c11f5ae9baa0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:37.437891  3460 consensus_queue.cc:1048] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:37.446789  3297 raft_consensus.cc:1275] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Refusing update from remote peer af2b734505a4432e93c11f5ae9baa0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:37.450284  3460 consensus_queue.cc:1048] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:37.458117  3448 raft_consensus.cc:993] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db: : Instructing follower db0bf45c4990494892675b3d9ff0bb5d to start an election
I20260709 12:22:37.458648  3462 raft_consensus.cc:1081] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 LEADER]: Signalling peer db0bf45c4990494892675b3d9ff0bb5d to start an election
I20260709 12:22:37.460573  3297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f"
dest_uuid: "db0bf45c4990494892675b3d9ff0bb5d"
 from {username='slave'} at 127.0.0.1:59924
I20260709 12:22:37.461145  3297 raft_consensus.cc:493] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:37.461555  3297 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.466745  3297 raft_consensus.cc:515] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:37.469707  3297 leader_election.cc:290] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [CANDIDATE]: Term 3 election: Requested vote from peers af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689), d6e3cf60df4d456fadd8de78cf7b6f10 (127.23.217.67:34753)
I20260709 12:22:37.483239  3370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "db0bf45c4990494892675b3d9ff0bb5d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10"
I20260709 12:22:37.483254  3222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d0ad633342440f85cd87b29fc4eb1f" candidate_uuid: "db0bf45c4990494892675b3d9ff0bb5d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af2b734505a4432e93c11f5ae9baa0db"
I20260709 12:22:37.483793  3370 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.484217  3222 raft_consensus.cc:3055] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:37.484575  3222 raft_consensus.cc:740] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: af2b734505a4432e93c11f5ae9baa0db, State: Running, Role: LEADER
I20260709 12:22:37.485407  3222 consensus_queue.cc:260] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:37.486644  3222 raft_consensus.cc:3060] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.489122  3370 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db0bf45c4990494892675b3d9ff0bb5d in term 3.
I20260709 12:22:37.490284  3261 leader_election.cc:304] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6e3cf60df4d456fadd8de78cf7b6f10, db0bf45c4990494892675b3d9ff0bb5d; no voters: 
I20260709 12:22:37.491115  3463 raft_consensus.cc:2804] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:37.491545  3463 raft_consensus.cc:697] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 3 LEADER]: Becoming Leader. State: Replica: db0bf45c4990494892675b3d9ff0bb5d, State: Running, Role: LEADER
I20260709 12:22:37.492261  3463 consensus_queue.cc:237] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } }
I20260709 12:22:37.493427  3222 raft_consensus.cc:2468] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db0bf45c4990494892675b3d9ff0bb5d in term 3.
I20260709 12:22:37.499079  3117 catalog_manager.cc:5697] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d reported cstate change: term changed from 2 to 3, leader changed from af2b734505a4432e93c11f5ae9baa0db (127.23.217.65) to db0bf45c4990494892675b3d9ff0bb5d (127.23.217.66). New cstate: current_term: 3 leader_uuid: "db0bf45c4990494892675b3d9ff0bb5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2b734505a4432e93c11f5ae9baa0db" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 45689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db0bf45c4990494892675b3d9ff0bb5d" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 41421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6e3cf60df4d456fadd8de78cf7b6f10" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:37.732585 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
I20260709 12:22:37.757928 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:37.758898 24421 tablet_replica.cc:333] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db: stopping tablet replica
I20260709 12:22:37.759649 24421 raft_consensus.cc:2243] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:37.760707 24421 raft_consensus.cc:2272] T c1905c80d100456581b937ce60c970d1 P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.763554 24421 tablet_replica.cc:333] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db: stopping tablet replica
I20260709 12:22:37.764183 24421 raft_consensus.cc:2243] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.764674 24421 raft_consensus.cc:2272] T 55d0ad633342440f85cd87b29fc4eb1f P af2b734505a4432e93c11f5ae9baa0db [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.767800 24421 tablet_replica.cc:333] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db: stopping tablet replica
I20260709 12:22:37.768419 24421 raft_consensus.cc:2243] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.768930 24421 raft_consensus.cc:2272] T 7a3ec06650d54d67a3d32b8ead1d136e P af2b734505a4432e93c11f5ae9baa0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.771641 24421 tablet_replica.cc:333] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db: stopping tablet replica
I20260709 12:22:37.772226 24421 raft_consensus.cc:2243] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:37.773180 24421 raft_consensus.cc:2272] T 8a085a1f703441c2adb789412b0e5319 P af2b734505a4432e93c11f5ae9baa0db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.799381 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
I20260709 12:22:37.814738 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
I20260709 12:22:37.845526 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:37.846482 24421 tablet_replica.cc:333] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: stopping tablet replica
I20260709 12:22:37.847283 24421 raft_consensus.cc:2243] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:37.848183 24421 pending_rounds.cc:70] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: Trying to abort 1 pending ops.
I20260709 12:22:37.848457 24421 pending_rounds.cc:77] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305624606695796736 op_type: NO_OP noop_request { }
I20260709 12:22:37.848807 24421 raft_consensus.cc:2889] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:22:37.849116 24421 raft_consensus.cc:2272] T 55d0ad633342440f85cd87b29fc4eb1f P db0bf45c4990494892675b3d9ff0bb5d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.851428 24421 tablet_replica.cc:333] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d: stopping tablet replica
I20260709 12:22:37.851970 24421 raft_consensus.cc:2243] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.852375 24421 raft_consensus.cc:2272] T c1905c80d100456581b937ce60c970d1 P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.854539 24421 tablet_replica.cc:333] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d: stopping tablet replica
I20260709 12:22:37.855064 24421 raft_consensus.cc:2243] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.855495 24421 raft_consensus.cc:2272] T 7a3ec06650d54d67a3d32b8ead1d136e P db0bf45c4990494892675b3d9ff0bb5d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.857602 24421 tablet_replica.cc:333] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d: stopping tablet replica
I20260709 12:22:37.858078 24421 raft_consensus.cc:2243] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.858474 24421 raft_consensus.cc:2272] T 8a085a1f703441c2adb789412b0e5319 P db0bf45c4990494892675b3d9ff0bb5d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:37.875838  3334 consensus_peers.cc:597] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 -> Peer af2b734505a4432e93c11f5ae9baa0db (127.23.217.65:45689): Couldn't send request to peer af2b734505a4432e93c11f5ae9baa0db. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:45689: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:37.882030 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
I20260709 12:22:37.896770 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
I20260709 12:22:37.920409 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:37.921248 24421 tablet_replica.cc:333] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10: stopping tablet replica
I20260709 12:22:37.921891 24421 raft_consensus.cc:2243] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.922434 24421 raft_consensus.cc:2272] T 55d0ad633342440f85cd87b29fc4eb1f P d6e3cf60df4d456fadd8de78cf7b6f10 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.925297 24421 tablet_replica.cc:333] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10: stopping tablet replica
I20260709 12:22:37.925887 24421 raft_consensus.cc:2243] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.926477 24421 raft_consensus.cc:2272] T c1905c80d100456581b937ce60c970d1 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.929071 24421 tablet_replica.cc:333] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10: stopping tablet replica
I20260709 12:22:37.929675 24421 raft_consensus.cc:2243] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:37.930576 24421 raft_consensus.cc:2272] T 7a3ec06650d54d67a3d32b8ead1d136e P d6e3cf60df4d456fadd8de78cf7b6f10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.933350 24421 tablet_replica.cc:333] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10: stopping tablet replica
I20260709 12:22:37.933872 24421 raft_consensus.cc:2243] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:37.934317 24421 raft_consensus.cc:2272] T 8a085a1f703441c2adb789412b0e5319 P d6e3cf60df4d456fadd8de78cf7b6f10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.958434 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
I20260709 12:22:37.972978 24421 master.cc:562] Master@127.23.217.126:34843 shutting down...
I20260709 12:22:37.991170 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:37.991796 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:37.992158 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P b75269998545475293d889b252b539d5: stopping tablet replica
I20260709 12:22:38.012586 24421 master.cc:584] Master@127.23.217.126:34843 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2758 ms)
[----------] 6 tests from LeaderRebalancerTest (395367 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:22:38.041178 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:41283
I20260709 12:22:38.042416 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:38.048509  3472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:38.048780  3473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:38.050256  3475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:38.053845 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:38.054801 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:38.054983 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:38.055116 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599758055106 us; error 0 us; skew 500 ppm
I20260709 12:22:38.055624 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:38.058029 24421 webserver.cc:533] Webserver started at http://127.23.217.126:41301/ using document root <none> and password file <none>
I20260709 12:22:38.058563 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:38.058723 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:38.058944 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:38.059947 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "c27bc8b6b87b4bd492fe58b423701fee"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.064302 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:38.067817  3480 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.068651 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:38.069088 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "c27bc8b6b87b4bd492fe58b423701fee"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.069429 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:38.091333 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:38.092337 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:38.131573 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:41283
I20260709 12:22:38.131742  3531 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:41283 every 8 connection(s)
I20260709 12:22:38.135807  3532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.149067  3532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee: Bootstrap starting.
I20260709 12:22:38.154373  3532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:38.159438  3532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee: No bootstrap required, opened a new log
I20260709 12:22:38.161836  3532 raft_consensus.cc:359] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bc8b6b87b4bd492fe58b423701fee" member_type: VOTER }
I20260709 12:22:38.162283  3532 raft_consensus.cc:385] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:38.162573  3532 raft_consensus.cc:740] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27bc8b6b87b4bd492fe58b423701fee, State: Initialized, Role: FOLLOWER
I20260709 12:22:38.163283  3532 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bc8b6b87b4bd492fe58b423701fee" member_type: VOTER }
I20260709 12:22:38.163733  3532 raft_consensus.cc:399] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:38.164016  3532 raft_consensus.cc:493] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:38.164324  3532 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:38.170360  3532 raft_consensus.cc:515] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bc8b6b87b4bd492fe58b423701fee" member_type: VOTER }
I20260709 12:22:38.171104  3532 leader_election.cc:304] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c27bc8b6b87b4bd492fe58b423701fee; no voters: 
I20260709 12:22:38.172432  3532 leader_election.cc:290] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:38.172996  3535 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:38.174479  3535 raft_consensus.cc:697] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 1 LEADER]: Becoming Leader. State: Replica: c27bc8b6b87b4bd492fe58b423701fee, State: Running, Role: LEADER
I20260709 12:22:38.175225  3535 consensus_queue.cc:237] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bc8b6b87b4bd492fe58b423701fee" member_type: VOTER }
I20260709 12:22:38.176172  3532 sys_catalog.cc:565] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:38.177969  3536 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c27bc8b6b87b4bd492fe58b423701fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bc8b6b87b4bd492fe58b423701fee" member_type: VOTER } }
I20260709 12:22:38.178216  3537 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27bc8b6b87b4bd492fe58b423701fee. Latest consensus state: current_term: 1 leader_uuid: "c27bc8b6b87b4bd492fe58b423701fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bc8b6b87b4bd492fe58b423701fee" member_type: VOTER } }
I20260709 12:22:38.178705  3536 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:38.178727  3537 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:38.182473  3539 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:38.187217  3539 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:38.192803 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:38.197017  3539 catalog_manager.cc:1383] Generated new cluster ID: bd95232aa6434d5cb4422dadec8f7695
I20260709 12:22:38.197310  3539 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:38.223838  3539 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:38.225179  3539 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:38.239440  3539 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee: Generated new TSK 0
I20260709 12:22:38.240056  3539 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:38.260735 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:38.267455  3553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:38.268728  3554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:38.272316  3556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:38.272722 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:38.273729 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:38.273910 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:38.274133 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599758274099 us; error 0 us; skew 500 ppm
I20260709 12:22:38.274784 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:38.277158 24421 webserver.cc:533] Webserver started at http://127.23.217.65:32911/ using document root <none> and password file <none>
I20260709 12:22:38.277661 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:38.277866 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:38.278193 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:38.279381 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "602b6ed4efe04787936c1f77080c61e6"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.283876 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:38.287441  3561 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.288301 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:38.288641 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "602b6ed4efe04787936c1f77080c61e6"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.288883 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:38.316058 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:38.317485 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:38.319144 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:38.321379 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:38.321593 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.321868 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:38.322052 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.365077 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:43629
I20260709 12:22:38.365255  3623 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:43629 every 8 connection(s)
I20260709 12:22:38.370088 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:38.377122  3628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:38.378006  3629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:38.387323 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:38.387729  3624 heartbeater.cc:344] Connected to a master server at 127.23.217.126:41283
I20260709 12:22:38.388516  3624 heartbeater.cc:461] Registering TS with master...
W20260709 12:22:38.388813  3631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:38.390028  3624 heartbeater.cc:507] Master 127.23.217.126:41283 requested a full tablet report, sending...
I20260709 12:22:38.392446 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:38.392771 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:38.392988 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599758392965 us; error 0 us; skew 500 ppm
I20260709 12:22:38.392988  3497 ts_manager.cc:194] Registered new tserver with Master: 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629)
I20260709 12:22:38.394136 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:38.395678  3497 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41314
I20260709 12:22:38.397315 24421 webserver.cc:533] Webserver started at http://127.23.217.66:37347/ using document root <none> and password file <none>
I20260709 12:22:38.398012 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:38.398245 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:38.398609 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:38.399706 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "522b9a515e00491e9c1dbc9a27ac099c"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.404210 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:22:38.407979  3636 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.408999 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:22:38.409325 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "522b9a515e00491e9c1dbc9a27ac099c"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.409626 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:38.427879 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:38.429128 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:38.430642 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:38.432890 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:38.433135 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.433375 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:38.433547 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.475251 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:40139
I20260709 12:22:38.475401  3698 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:40139 every 8 connection(s)
I20260709 12:22:38.480242 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:38.487987  3702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:38.489921  3703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:38.492342 24421 server_base.cc:1061] running on GCE node
W20260709 12:22:38.492455  3705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:38.493650 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:38.493927 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:38.494115 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599758494099 us; error 0 us; skew 500 ppm
I20260709 12:22:38.494791 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:38.495919  3699 heartbeater.cc:344] Connected to a master server at 127.23.217.126:41283
I20260709 12:22:38.496345  3699 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:38.497243  3699 heartbeater.cc:507] Master 127.23.217.126:41283 requested a full tablet report, sending...
I20260709 12:22:38.497687 24421 webserver.cc:533] Webserver started at http://127.23.217.67:43829/ using document root <none> and password file <none>
I20260709 12:22:38.498339 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:38.498598 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:38.498935 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:38.499733  3497 ts_manager.cc:194] Registered new tserver with Master: 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:38.500065 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "915f522cefa74142aa1c5914a877dbbf"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.501204  3497 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41318
I20260709 12:22:38.505265 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:22:38.509256  3710 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.510087 24421 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:22:38.510421 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "915f522cefa74142aa1c5914a877dbbf"
format_stamp: "Formatted at 2026-07-09 12:22:38 on dist-test-slave-lg8f"
I20260709 12:22:38.510767 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:38.524300 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:38.525236 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:38.526639 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:38.528805 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:38.529034 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.529282 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:38.529454 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:38.571949 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:34829
I20260709 12:22:38.572052  3772 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:34829 every 8 connection(s)
I20260709 12:22:38.587145  3773 heartbeater.cc:344] Connected to a master server at 127.23.217.126:41283
I20260709 12:22:38.587589  3773 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:38.588445  3773 heartbeater.cc:507] Master 127.23.217.126:41283 requested a full tablet report, sending...
I20260709 12:22:38.590672  3497 ts_manager.cc:194] Registered new tserver with Master: 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:38.591193 24421 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015911607s
I20260709 12:22:38.592224  3497 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41324
I20260709 12:22:38.618901  3497 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41336:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:38.621739  3497 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:38.694761  3589 tablet_service.cc:1511] Processing CreateTablet for tablet 6827650806d94c81b65b6bf932270220 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:38.696477  3589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6827650806d94c81b65b6bf932270220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.698704  3588 tablet_service.cc:1511] Processing CreateTablet for tablet 8df4615f2eae4a22b1dce75d1a544712 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:38.700304  3588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8df4615f2eae4a22b1dce75d1a544712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.712323  3738 tablet_service.cc:1511] Processing CreateTablet for tablet 6827650806d94c81b65b6bf932270220 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:38.712946  3737 tablet_service.cc:1511] Processing CreateTablet for tablet 8df4615f2eae4a22b1dce75d1a544712 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:38.713883  3738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6827650806d94c81b65b6bf932270220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.714529  3737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8df4615f2eae4a22b1dce75d1a544712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.716589  3587 tablet_service.cc:1511] Processing CreateTablet for tablet ce0ea8bd86f245ad8264344eb0fe2c92 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:38.717933  3587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce0ea8bd86f245ad8264344eb0fe2c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.721359  3736 tablet_service.cc:1511] Processing CreateTablet for tablet ce0ea8bd86f245ad8264344eb0fe2c92 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:38.722801  3736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce0ea8bd86f245ad8264344eb0fe2c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.725819  3664 tablet_service.cc:1511] Processing CreateTablet for tablet 6827650806d94c81b65b6bf932270220 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:38.727476  3664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6827650806d94c81b65b6bf932270220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.727761  3586 tablet_service.cc:1511] Processing CreateTablet for tablet 4da3e8ce02cc4f2589ce772d40d87fbc (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:38.729377  3586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da3e8ce02cc4f2589ce772d40d87fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.732664  3585 tablet_service.cc:1511] Processing CreateTablet for tablet 35d9ba63cc334a0d960ebcf224c79756 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:38.734529  3585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d9ba63cc334a0d960ebcf224c79756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.734745  3663 tablet_service.cc:1511] Processing CreateTablet for tablet 8df4615f2eae4a22b1dce75d1a544712 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:38.732758  3735 tablet_service.cc:1511] Processing CreateTablet for tablet 4da3e8ce02cc4f2589ce772d40d87fbc (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:38.736739  3663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8df4615f2eae4a22b1dce75d1a544712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.737116  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da3e8ce02cc4f2589ce772d40d87fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.765744  3734 tablet_service.cc:1511] Processing CreateTablet for tablet 35d9ba63cc334a0d960ebcf224c79756 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:38.767457  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d9ba63cc334a0d960ebcf224c79756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.768916  3662 tablet_service.cc:1511] Processing CreateTablet for tablet ce0ea8bd86f245ad8264344eb0fe2c92 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:38.770450  3662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce0ea8bd86f245ad8264344eb0fe2c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.781546  3661 tablet_service.cc:1511] Processing CreateTablet for tablet 4da3e8ce02cc4f2589ce772d40d87fbc (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:38.783174  3661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da3e8ce02cc4f2589ce772d40d87fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.790743  3584 tablet_service.cc:1511] Processing CreateTablet for tablet 086558ab35914c12869f94ad34540292 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:38.792215  3584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086558ab35914c12869f94ad34540292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.793260  3660 tablet_service.cc:1511] Processing CreateTablet for tablet 35d9ba63cc334a0d960ebcf224c79756 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:38.795210  3660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d9ba63cc334a0d960ebcf224c79756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.798216  3733 tablet_service.cc:1511] Processing CreateTablet for tablet 086558ab35914c12869f94ad34540292 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:38.799860  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086558ab35914c12869f94ad34540292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.805176  3659 tablet_service.cc:1511] Processing CreateTablet for tablet 086558ab35914c12869f94ad34540292 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:38.806622  3788 tablet_bootstrap.cc:492] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:38.806903  3659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086558ab35914c12869f94ad34540292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.813465  3788 tablet_bootstrap.cc:654] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:38.814891  3589 tablet_service.cc:1511] Processing CreateTablet for tablet d1ccf46e805348be875b04ab724ec30e (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:38.816354  3589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ccf46e805348be875b04ab724ec30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.827726  3658 tablet_service.cc:1511] Processing CreateTablet for tablet d1ccf46e805348be875b04ab724ec30e (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:38.829275  3658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ccf46e805348be875b04ab724ec30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.832103  3732 tablet_service.cc:1511] Processing CreateTablet for tablet d1ccf46e805348be875b04ab724ec30e (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:38.833712  3732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ccf46e805348be875b04ab724ec30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.850960  3584 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4852bb48c749abb6c9c2bc6c786d02 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:38.852588  3584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4852bb48c749abb6c9c2bc6c786d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.857276  3790 tablet_bootstrap.cc:492] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:38.864100  3790 tablet_bootstrap.cc:654] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:38.868903  3657 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4852bb48c749abb6c9c2bc6c786d02 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:38.870515  3657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4852bb48c749abb6c9c2bc6c786d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.886518  3735 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4852bb48c749abb6c9c2bc6c786d02 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:38.888118  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4852bb48c749abb6c9c2bc6c786d02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.899698  3792 tablet_bootstrap.cc:492] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:38.900684  3788 tablet_bootstrap.cc:492] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:38.901330  3788 ts_tablet_manager.cc:1403] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.095s	user 0.018s	sys 0.018s
I20260709 12:22:38.906244  3788 raft_consensus.cc:359] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.907114  3788 raft_consensus.cc:385] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:38.907529  3788 raft_consensus.cc:740] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:38.908403  3788 consensus_queue.cc:260] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.913424  3585 tablet_service.cc:1511] Processing CreateTablet for tablet f78d8d43e1af454e9be9691159fa7525 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:38.915498  3585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78d8d43e1af454e9be9691159fa7525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.918948  3788 ts_tablet_manager.cc:1434] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.006s
I20260709 12:22:38.919914  3624 heartbeater.cc:499] Master 127.23.217.126:41283 was elected leader, sending a full tablet report...
I20260709 12:22:38.927489  3788 tablet_bootstrap.cc:492] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:38.935015  3788 tablet_bootstrap.cc:654] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:38.946601  3792 tablet_bootstrap.cc:654] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:38.950030  3663 tablet_service.cc:1511] Processing CreateTablet for tablet f78d8d43e1af454e9be9691159fa7525 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:38.951380  3663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78d8d43e1af454e9be9691159fa7525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.955099  3788 tablet_bootstrap.cc:492] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:38.955687  3788 ts_tablet_manager.cc:1403] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.001s
I20260709 12:22:38.955446  3732 tablet_service.cc:1511] Processing CreateTablet for tablet f78d8d43e1af454e9be9691159fa7525 (DEFAULT_TABLE table=test-workload [id=1831ab2e721a44f586e920baeaecaf59]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:38.956604  3790 tablet_bootstrap.cc:492] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:38.956710  3732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78d8d43e1af454e9be9691159fa7525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:38.957645  3790 ts_tablet_manager.cc:1403] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.101s	user 0.032s	sys 0.054s
I20260709 12:22:38.958683  3788 raft_consensus.cc:359] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.959457  3788 raft_consensus.cc:385] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:38.959757  3788 raft_consensus.cc:740] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:38.960623  3788 consensus_queue.cc:260] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.961376  3790 raft_consensus.cc:359] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.962255  3790 raft_consensus.cc:385] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:38.962636  3790 raft_consensus.cc:740] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:38.962793  3788 ts_tablet_manager.cc:1434] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:38.964075  3788 tablet_bootstrap.cc:492] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:38.963663  3790 consensus_queue.cc:260] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.972324  3788 tablet_bootstrap.cc:654] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:38.987603  3792 tablet_bootstrap.cc:492] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:38.988272  3792 ts_tablet_manager.cc:1403] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.004s
I20260709 12:22:38.992273  3790 ts_tablet_manager.cc:1434] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.034s	user 0.022s	sys 0.009s
I20260709 12:22:38.993252  3792 raft_consensus.cc:359] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.994067  3792 raft_consensus.cc:385] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:38.994362  3792 raft_consensus.cc:740] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:38.994977  3773 heartbeater.cc:499] Master 127.23.217.126:41283 was elected leader, sending a full tablet report...
I20260709 12:22:38.995863  3792 consensus_queue.cc:260] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:38.999082  3699 heartbeater.cc:499] Master 127.23.217.126:41283 was elected leader, sending a full tablet report...
I20260709 12:22:38.999696  3792 ts_tablet_manager.cc:1434] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:22:38.999935  3790 tablet_bootstrap.cc:492] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.000887  3792 tablet_bootstrap.cc:492] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.000901  3788 tablet_bootstrap.cc:492] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:39.001662  3788 ts_tablet_manager.cc:1403] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.038s	user 0.022s	sys 0.014s
I20260709 12:22:39.004956  3788 raft_consensus.cc:359] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.005802  3788 raft_consensus.cc:385] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.006105  3788 raft_consensus.cc:740] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.007014  3788 consensus_queue.cc:260] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.007833  3792 tablet_bootstrap.cc:654] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.008118  3790 tablet_bootstrap.cc:654] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.012871  3788 ts_tablet_manager.cc:1434] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:22:39.014657  3788 tablet_bootstrap.cc:492] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:39.022046  3788 tablet_bootstrap.cc:654] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.055629  3788 tablet_bootstrap.cc:492] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:39.055714  3790 tablet_bootstrap.cc:492] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.055650  3792 tablet_bootstrap.cc:492] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.056319  3790 ts_tablet_manager.cc:1403] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.057s	user 0.024s	sys 0.019s
I20260709 12:22:39.056319  3788 ts_tablet_manager.cc:1403] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.042s	user 0.024s	sys 0.015s
I20260709 12:22:39.056681  3792 ts_tablet_manager.cc:1403] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.056s	user 0.009s	sys 0.037s
I20260709 12:22:39.059171  3792 raft_consensus.cc:359] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.059767  3792 raft_consensus.cc:385] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.060071  3792 raft_consensus.cc:740] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.059844  3788 raft_consensus.cc:359] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.060012  3790 raft_consensus.cc:359] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.060838  3788 raft_consensus.cc:385] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.060976  3790 raft_consensus.cc:385] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.061339  3790 raft_consensus.cc:740] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.061188  3792 consensus_queue.cc:260] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.062701  3790 consensus_queue.cc:260] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.063753  3792 ts_tablet_manager.cc:1434] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:39.061225  3788 raft_consensus.cc:740] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.064828  3792 tablet_bootstrap.cc:492] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.065982  3788 consensus_queue.cc:260] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.068461  3788 ts_tablet_manager.cc:1434] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:22:39.069613  3788 tablet_bootstrap.cc:492] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:39.072045  3792 tablet_bootstrap.cc:654] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.064855  3790 ts_tablet_manager.cc:1434] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:39.074707  3788 tablet_bootstrap.cc:654] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.078286  3790 tablet_bootstrap.cc:492] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.079187  3793 raft_consensus.cc:493] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.079923  3793 raft_consensus.cc:515] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.083536  3792 tablet_bootstrap.cc:492] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.084122  3792 ts_tablet_manager.cc:1403] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 12:22:39.085552  3793 leader_election.cc:290] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.088629  3790 tablet_bootstrap.cc:654] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.094316  3792 raft_consensus.cc:359] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.095116  3792 raft_consensus.cc:385] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.095481  3792 raft_consensus.cc:740] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.104238  3792 consensus_queue.cc:260] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.106423  3792 ts_tablet_manager.cc:1434] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:22:39.107749  3792 tablet_bootstrap.cc:492] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.118075  3788 tablet_bootstrap.cc:492] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:39.119002  3788 ts_tablet_manager.cc:1403] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.024s
I20260709 12:22:39.122011  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.122788  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
W20260709 12:22:39.123624  3563 leader_election.cc:343] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:39.124421  3565 leader_election.cc:343] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.124876  3565 leader_election.cc:304] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6; no voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:39.125669  3793 raft_consensus.cc:2749] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:39.125757  3788 raft_consensus.cc:359] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.126801  3788 raft_consensus.cc:385] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.127115  3788 raft_consensus.cc:740] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.128057  3790 tablet_bootstrap.cc:492] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.127954  3788 consensus_queue.cc:260] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.128793  3792 tablet_bootstrap.cc:654] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.130566  3788 ts_tablet_manager.cc:1434] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:39.128556  3790 ts_tablet_manager.cc:1403] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.051s	user 0.027s	sys 0.018s
I20260709 12:22:39.131882  3788 tablet_bootstrap.cc:492] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:39.135192  3790 raft_consensus.cc:359] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.135975  3790 raft_consensus.cc:385] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.136277  3792 tablet_bootstrap.cc:492] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.136297  3790 raft_consensus.cc:740] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.137007  3792 ts_tablet_manager.cc:1403] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.000s
I20260709 12:22:39.137823  3790 consensus_queue.cc:260] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.139189  3788 tablet_bootstrap.cc:654] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.140199  3792 raft_consensus.cc:359] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.140985  3792 raft_consensus.cc:385] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.141310  3792 raft_consensus.cc:740] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.142225  3792 consensus_queue.cc:260] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.143324  3790 ts_tablet_manager.cc:1434] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 12:22:39.144865  3790 tablet_bootstrap.cc:492] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.147220  3792 ts_tablet_manager.cc:1434] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:22:39.148270  3792 tablet_bootstrap.cc:492] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.154348  3790 tablet_bootstrap.cc:654] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.155228  3792 tablet_bootstrap.cc:654] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.161422  3788 tablet_bootstrap.cc:492] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:39.162042  3788 ts_tablet_manager.cc:1403] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.011s
I20260709 12:22:39.164978  3792 tablet_bootstrap.cc:492] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.165046  3790 tablet_bootstrap.cc:492] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.165858  3792 ts_tablet_manager.cc:1403] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.001s
I20260709 12:22:39.165938  3790 ts_tablet_manager.cc:1403] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.013s
I20260709 12:22:39.166019  3788 raft_consensus.cc:359] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.167765  3788 raft_consensus.cc:385] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.168079  3788 raft_consensus.cc:740] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.169030  3788 consensus_queue.cc:260] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.169847  3790 raft_consensus.cc:359] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.170594  3790 raft_consensus.cc:385] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.170264  3792 raft_consensus.cc:359] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.170926  3790 raft_consensus.cc:740] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.171135  3792 raft_consensus.cc:385] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.171547  3792 raft_consensus.cc:740] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.171808  3790 consensus_queue.cc:260] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.173660  3788 ts_tablet_manager.cc:1434] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.002s
I20260709 12:22:39.174289  3790 ts_tablet_manager.cc:1434] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 12:22:39.174888  3788 tablet_bootstrap.cc:492] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:39.175454  3790 tablet_bootstrap.cc:492] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.172801  3792 consensus_queue.cc:260] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.180941  3792 ts_tablet_manager.cc:1434] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:22:39.182163  3792 tablet_bootstrap.cc:492] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.182898  3790 tablet_bootstrap.cc:654] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.183092  3788 tablet_bootstrap.cc:654] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.190647  3792 tablet_bootstrap.cc:654] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:39.192034  3545 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:39.193075  3798 raft_consensus.cc:493] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.193598  3798 raft_consensus.cc:515] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.194626  3790 tablet_bootstrap.cc:492] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.195130  3790 ts_tablet_manager.cc:1403] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 12:22:39.195660  3788 tablet_bootstrap.cc:492] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:39.196413  3788 ts_tablet_manager.cc:1403] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:22:39.198575  3790 raft_consensus.cc:359] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.199384  3790 raft_consensus.cc:385] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.199680  3790 raft_consensus.cc:740] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.204803  3798 leader_election.cc:290] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.217744  3798 raft_consensus.cc:493] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.218508  3798 raft_consensus.cc:515] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.223093  3798 leader_election.cc:290] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.230293  3796 raft_consensus.cc:493] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.231043  3796 raft_consensus.cc:515] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.219113  3790 consensus_queue.cc:260] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.230608  3788 raft_consensus.cc:359] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.241323  3788 raft_consensus.cc:385] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.241675  3788 raft_consensus.cc:740] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.242530  3788 consensus_queue.cc:260] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.244376  3792 tablet_bootstrap.cc:492] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.245066  3788 ts_tablet_manager.cc:1434] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.002s
I20260709 12:22:39.245361  3792 ts_tablet_manager.cc:1403] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.063s	user 0.018s	sys 0.026s
I20260709 12:22:39.246235  3788 tablet_bootstrap.cc:492] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:39.247917  3796 leader_election.cc:290] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:39.258913  3793 raft_consensus.cc:493] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.266144  3793 raft_consensus.cc:515] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.268380  3793 leader_election.cc:290] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.269713  3788 tablet_bootstrap.cc:654] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.269832  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.256621  3792 raft_consensus.cc:359] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.276767  3792 raft_consensus.cc:385] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.277207  3793 raft_consensus.cc:493] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.277446  3792 raft_consensus.cc:740] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.277849  3793 raft_consensus.cc:515] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.272410  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:39.278426  3792 consensus_queue.cc:260] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.279134  3674 raft_consensus.cc:2468] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:39.270706  3748 raft_consensus.cc:2468] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:39.280382  3565 leader_election.cc:304] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.283608  3793 leader_election.cc:290] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.284330  3793 raft_consensus.cc:2804] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.284775  3793 raft_consensus.cc:493] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.285177  3793 raft_consensus.cc:3060] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.281734  3792 ts_tablet_manager.cc:1434] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.004s
I20260709 12:22:39.288003  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4852bb48c749abb6c9c2bc6c786d02" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
W20260709 12:22:39.290592  3563 leader_election.cc:343] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.299222  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.299885  3790 ts_tablet_manager.cc:1434] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.104s	user 0.014s	sys 0.012s
I20260709 12:22:39.290148  3792 tablet_bootstrap.cc:492] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.300616  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.301132  3790 tablet_bootstrap.cc:492] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.304103  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.305207  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
W20260709 12:22:39.305877  3638 leader_election.cc:343] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.305977  3748 raft_consensus.cc:2468] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:39.307793  3638 leader_election.cc:304] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
W20260709 12:22:39.308091  3639 leader_election.cc:343] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.308617  3639 leader_election.cc:304] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c; no voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:39.309183  3815 raft_consensus.cc:2804] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.309636  3815 raft_consensus.cc:493] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.309898  3798 raft_consensus.cc:2749] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:39.309998  3815 raft_consensus.cc:3060] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.311867  3793 raft_consensus.cc:515] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.315073  3599 raft_consensus.cc:2393] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522b9a515e00491e9c1dbc9a27ac099c in current term 1: Already voted for candidate 602b6ed4efe04787936c1f77080c61e6 in this term.
I20260709 12:22:39.315416  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:39.316120  3798 raft_consensus.cc:493] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.316382  3674 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 0.
I20260709 12:22:39.317037  3798 raft_consensus.cc:515] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.318131  3714 leader_election.cc:304] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.316179  3793 leader_election.cc:290] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.319325  3818 raft_consensus.cc:2804] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.319411  3790 tablet_bootstrap.cc:654] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.320873  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.321556  3748 raft_consensus.cc:3060] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.322010  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.322886  3599 raft_consensus.cc:2468] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:39.324242  3639 leader_election.cc:304] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.319521  3798 leader_election.cc:290] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.319947  3818 raft_consensus.cc:493] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.326409  3798 raft_consensus.cc:2804] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.326673  3818 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.326853  3798 raft_consensus.cc:493] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.327272  3798 raft_consensus.cc:3060] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.329025  3793 raft_consensus.cc:493] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.329602  3793 raft_consensus.cc:515] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.333938  3793 leader_election.cc:290] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.334178  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce0ea8bd86f245ad8264344eb0fe2c92" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.335047  3747 raft_consensus.cc:2468] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:39.336265  3563 leader_election.cc:304] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.337487  3793 raft_consensus.cc:2804] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.337976  3793 raft_consensus.cc:493] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.338301  3793 raft_consensus.cc:3060] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:39.338315  3565 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:40139, user_credentials={real_user=slave}} blocked reactor thread for 58023.6us
I20260709 12:22:39.340714  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4852bb48c749abb6c9c2bc6c786d02" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:39.344336  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:39.346714  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce0ea8bd86f245ad8264344eb0fe2c92" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:39.347329  3796 raft_consensus.cc:493] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.347553  3673 raft_consensus.cc:2468] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:39.347908  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.348707  3747 raft_consensus.cc:2468] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:39.348315  3796 raft_consensus.cc:515] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
W20260709 12:22:39.350925  3565 leader_election.cc:343] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.351038  3796 leader_election.cc:290] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:39.352540  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
W20260709 12:22:39.353884  3714 leader_election.cc:343] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.355420  3565 leader_election.cc:304] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6; no voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:39.356601  3822 raft_consensus.cc:2749] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:39.357208  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.357972  3599 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 0.
I20260709 12:22:39.359508  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.360268  3598 raft_consensus.cc:2468] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 0.
I20260709 12:22:39.362320  3713 leader_election.cc:304] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:39.362468  3792 tablet_bootstrap.cc:654] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.363272  3796 raft_consensus.cc:2804] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.375543  3796 raft_consensus.cc:493] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.375934  3796 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.372399  3818 raft_consensus.cc:515] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.379612  3788 tablet_bootstrap.cc:492] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:39.380499  3788 ts_tablet_manager.cc:1403] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.134s	user 0.027s	sys 0.041s
I20260709 12:22:39.370400  3793 raft_consensus.cc:515] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.382159  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:39.382899  3673 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.383448  3793 leader_election.cc:290] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.381381  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:39.373476  3825 raft_consensus.cc:493] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.384262  3598 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.384654  3825 raft_consensus.cc:515] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.385197  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce0ea8bd86f245ad8264344eb0fe2c92" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.386132  3747 raft_consensus.cc:3060] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.366951  3815 raft_consensus.cc:515] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.388222  3674 raft_consensus.cc:2393] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 602b6ed4efe04787936c1f77080c61e6 in current term 1: Already voted for candidate 522b9a515e00491e9c1dbc9a27ac099c in this term.
I20260709 12:22:39.390134  3815 leader_election.cc:290] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.374751  3748 raft_consensus.cc:2468] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:39.391752  3788 raft_consensus.cc:359] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.392174  3563 leader_election.cc:304] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.392901  3818 leader_election.cc:290] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:39.395748  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.396652  3748 raft_consensus.cc:2393] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 522b9a515e00491e9c1dbc9a27ac099c in current term 1: Already voted for candidate 602b6ed4efe04787936c1f77080c61e6 in this term.
I20260709 12:22:39.397816  3825 leader_election.cc:290] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.398779  3793 raft_consensus.cc:2804] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.399410  3793 raft_consensus.cc:697] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:39.400368  3793 consensus_queue.cc:237] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.365130  3798 raft_consensus.cc:515] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.400825  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce0ea8bd86f245ad8264344eb0fe2c92" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.402421  3790 tablet_bootstrap.cc:492] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.403097  3790 ts_tablet_manager.cc:1403] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.102s	user 0.024s	sys 0.055s
I20260709 12:22:39.404392  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce0ea8bd86f245ad8264344eb0fe2c92" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:39.405061  3674 raft_consensus.cc:3060] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.407434  3792 tablet_bootstrap.cc:492] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.407792  3747 raft_consensus.cc:2468] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:39.408002  3792 ts_tablet_manager.cc:1403] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.118s	user 0.010s	sys 0.004s
I20260709 12:22:39.408800  3788 raft_consensus.cc:385] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.409150  3788 raft_consensus.cc:740] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.409265  3563 leader_election.cc:304] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.410631  3822 raft_consensus.cc:2804] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.410524  3788 consensus_queue.cc:260] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.408825  3790 raft_consensus.cc:359] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.411954  3790 raft_consensus.cc:385] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.422060  3792 raft_consensus.cc:359] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.422690  3798 leader_election.cc:290] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.424026  3792 raft_consensus.cc:385] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.421397  3496 catalog_manager.cc:5697] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.423653  3790 raft_consensus.cc:740] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.426019  3790 consensus_queue.cc:260] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
W20260709 12:22:39.373319  3639 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:43629, user_credentials={real_user=slave}} blocked reactor thread for 49168.1us
I20260709 12:22:39.433073  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:39.435171  3792 raft_consensus.cc:740] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.435241  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.435979  3747 raft_consensus.cc:3060] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.438248  3792 consensus_queue.cc:260] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.439808  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce0ea8bd86f245ad8264344eb0fe2c92" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.442556  3597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:39.443313  3597 raft_consensus.cc:3060] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.444263  3792 ts_tablet_manager.cc:1434] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.003s
I20260709 12:22:39.445472  3792 tablet_bootstrap.cc:492] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.453644  3597 raft_consensus.cc:2468] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:39.462440  3788 ts_tablet_manager.cc:1434] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.082s	user 0.023s	sys 0.030s
I20260709 12:22:39.464042  3792 tablet_bootstrap.cc:654] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.477615  3747 raft_consensus.cc:2468] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:39.459043  3822 raft_consensus.cc:697] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:39.463555  3674 raft_consensus.cc:2468] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:39.458215  3673 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 1.
I20260709 12:22:39.457778  3790 ts_tablet_manager.cc:1434] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.003s
I20260709 12:22:39.459468  3639 leader_election.cc:304] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c; no voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:39.429283  3796 raft_consensus.cc:515] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.479188  3822 consensus_queue.cc:237] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.461038  3598 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 1.
I20260709 12:22:39.481961  3713 leader_election.cc:304] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.482372  3798 raft_consensus.cc:2749] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:39.483028  3818 raft_consensus.cc:2804] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.485328  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:39.486020  3796 leader_election.cc:290] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:39.486111  3598 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.461913  3829 raft_consensus.cc:493] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.479519  3638 leader_election.cc:304] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.487985  3798 raft_consensus.cc:2804] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.488534  3818 raft_consensus.cc:697] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:39.489462  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:39.490120  3673 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.489921  3818 consensus_queue.cc:237] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.488510  3798 raft_consensus.cc:697] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:39.493615  3798 consensus_queue.cc:237] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.497673  3598 raft_consensus.cc:2468] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 1.
I20260709 12:22:39.499147  3713 leader_election.cc:304] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.499805  3639 leader_election.cc:304] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c; no voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:39.499933  3673 raft_consensus.cc:2468] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 1.
I20260709 12:22:39.500129  3796 raft_consensus.cc:2804] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.500851  3825 raft_consensus.cc:2749] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:39.501055  3796 raft_consensus.cc:697] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:39.501893  3829 raft_consensus.cc:515] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.502249  3796 consensus_queue.cc:237] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.488008  3788 tablet_bootstrap.cc:492] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:39.505652  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:39.506629  3674 raft_consensus.cc:2468] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:39.507519  3798 raft_consensus.cc:493] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.508155  3565 leader_election.cc:304] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.508267  3798 raft_consensus.cc:515] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.508780  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.509385  3822 raft_consensus.cc:2804] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.510044  3822 raft_consensus.cc:493] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.510624  3822 raft_consensus.cc:3060] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:39.510911  3563 leader_election.cc:343] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.511817  3790 tablet_bootstrap.cc:492] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.512087  3829 leader_election.cc:290] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.518608  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.519096  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.519413  3598 raft_consensus.cc:2468] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:39.519874  3747 raft_consensus.cc:2468] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:39.520654  3822 raft_consensus.cc:515] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.520867  3639 leader_election.cc:304] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.521389  3798 leader_election.cc:290] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.522064  3798 raft_consensus.cc:2804] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.522424  3798 raft_consensus.cc:493] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.526700  3798 raft_consensus.cc:3060] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.524360  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.525071  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:39.528376  3674 raft_consensus.cc:3060] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:39.529294  3563 leader_election.cc:343] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:39.530194  3790 tablet_bootstrap.cc:654] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.524024  3822 leader_election.cc:290] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.532056  3788 tablet_bootstrap.cc:654] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.531060  3494 catalog_manager.cc:5697] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 0 to 1, leader changed from <none> to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 1 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.537364  3792 tablet_bootstrap.cc:492] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.538100  3792 ts_tablet_manager.cc:1403] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.093s	user 0.025s	sys 0.011s
I20260709 12:22:39.550613  3792 raft_consensus.cc:359] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.551753  3792 raft_consensus.cc:385] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.552158  3792 raft_consensus.cc:740] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.553516  3790 tablet_bootstrap.cc:492] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.552302  3798 raft_consensus.cc:515] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.553279  3792 consensus_queue.cc:260] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.555560  3790 ts_tablet_manager.cc:1403] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.012s
I20260709 12:22:39.560395  3798 leader_election.cc:290] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.560721  3792 ts_tablet_manager.cc:1434] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 12:22:39.561772  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:39.562232  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.562500  3598 raft_consensus.cc:3060] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.562875  3747 raft_consensus.cc:3060] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.562242  3792 tablet_bootstrap.cc:492] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:39.567003  3788 tablet_bootstrap.cc:492] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:39.567492  3788 ts_tablet_manager.cc:1403] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.080s	user 0.027s	sys 0.014s
I20260709 12:22:39.570254  3788 raft_consensus.cc:359] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.570945  3788 raft_consensus.cc:385] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.571287  3788 raft_consensus.cc:740] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.571353  3747 raft_consensus.cc:2468] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:39.572374  3788 consensus_queue.cc:260] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.572898  3638 leader_election.cc:304] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:39.573355  3792 tablet_bootstrap.cc:654] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.573895  3798 raft_consensus.cc:2804] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.571007  3598 raft_consensus.cc:2468] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:39.574537  3788 ts_tablet_manager.cc:1434] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:39.574569  3798 raft_consensus.cc:697] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:39.576005  3798 consensus_queue.cc:237] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.579104  3792 tablet_bootstrap.cc:492] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:39.579619  3792 ts_tablet_manager.cc:1403] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.000s
I20260709 12:22:39.557708  3674 raft_consensus.cc:2468] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:39.581372  3565 leader_election.cc:304] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:39.582249  3822 raft_consensus.cc:2804] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.582362  3792 raft_consensus.cc:359] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.582856  3822 raft_consensus.cc:697] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:39.583226  3792 raft_consensus.cc:385] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.583598  3792 raft_consensus.cc:740] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.584018  3790 raft_consensus.cc:359] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.584373  3792 consensus_queue.cc:260] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.584863  3790 raft_consensus.cc:385] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.584530  3822 consensus_queue.cc:237] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.586551  3792 ts_tablet_manager.cc:1434] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:39.585191  3790 raft_consensus.cc:740] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.589802  3790 consensus_queue.cc:260] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.592653  3790 ts_tablet_manager.cc:1434] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:22:39.593838  3790 tablet_bootstrap.cc:492] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.601177  3497 catalog_manager.cc:5697] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 1 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:39.603330  3497 catalog_manager.cc:5697] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 1 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:39.606580  3494 catalog_manager.cc:5697] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.607281  3790 tablet_bootstrap.cc:654] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.608106  3494 catalog_manager.cc:5697] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.613795  3790 tablet_bootstrap.cc:492] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.614611  3790 ts_tablet_manager.cc:1403] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.006s
I20260709 12:22:39.617086  3790 raft_consensus.cc:359] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.617748  3790 raft_consensus.cc:385] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.618011  3790 raft_consensus.cc:740] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.618731  3790 consensus_queue.cc:260] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.620967  3790 ts_tablet_manager.cc:1434] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:39.620460  3495 catalog_manager.cc:5697] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 0 to 1, leader changed from <none> to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 1 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.622112  3790 tablet_bootstrap.cc:492] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:39.633581  3790 tablet_bootstrap.cc:654] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:39.638633  3790 tablet_bootstrap.cc:492] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:39.639112  3790 ts_tablet_manager.cc:1403] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.003s
I20260709 12:22:39.641366  3790 raft_consensus.cc:359] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.641837  3790 raft_consensus.cc:385] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:39.642108  3790 raft_consensus.cc:740] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:39.642735  3790 consensus_queue.cc:260] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.644407  3790 ts_tablet_manager.cc:1434] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 12:22:39.820093  3822 raft_consensus.cc:493] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.820827  3822 raft_consensus.cc:515] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.822803  3822 leader_election.cc:290] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.823870  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:39.823995  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.824709  3674 raft_consensus.cc:2468] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:39.824798  3747 raft_consensus.cc:2468] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:39.825928  3565 leader_election.cc:304] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.826732  3822 raft_consensus.cc:2804] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.827082  3822 raft_consensus.cc:493] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.827358  3822 raft_consensus.cc:3060] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.833164  3822 raft_consensus.cc:515] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.835381  3822 leader_election.cc:290] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.836356  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:39.836731  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.837080  3674 raft_consensus.cc:3060] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.837558  3747 raft_consensus.cc:3060] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.845811  3674 raft_consensus.cc:2468] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:39.846001  3747 raft_consensus.cc:2468] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:39.847198  3565 leader_election.cc:304] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.848016  3822 raft_consensus.cc:2804] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.848500  3822 raft_consensus.cc:697] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:39.849391  3822 consensus_queue.cc:237] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.853948  3822 consensus_queue.cc:1048] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:39.858451  3497 catalog_manager.cc:5697] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.868311  3828 consensus_queue.cc:1048] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:39.921205  3798 consensus_queue.cc:1048] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:39.932708  3825 consensus_queue.cc:1048] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:39.957211  3825 raft_consensus.cc:493] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.957731  3825 raft_consensus.cc:515] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.959578  3825 leader_election.cc:290] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.960536  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4852bb48c749abb6c9c2bc6c786d02" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:39.960819  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4852bb48c749abb6c9c2bc6c786d02" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:39.961290  3598 raft_consensus.cc:2468] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:39.961704  3747 raft_consensus.cc:2468] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:39.962544  3639 leader_election.cc:304] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.963464  3825 raft_consensus.cc:2804] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:39.963852  3825 raft_consensus.cc:493] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.964082  3825 raft_consensus.cc:3060] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.969689  3825 raft_consensus.cc:515] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.971680  3825 leader_election.cc:290] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:39.972381  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4852bb48c749abb6c9c2bc6c786d02" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:39.972677  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4852bb48c749abb6c9c2bc6c786d02" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:39.973111  3598 raft_consensus.cc:3060] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.973273  3747 raft_consensus.cc:3060] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:39.981284  3598 raft_consensus.cc:2468] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:39.981313  3747 raft_consensus.cc:2468] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:39.982853  3639 leader_election.cc:304] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:39.983774  3825 raft_consensus.cc:2804] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:39.984416  3825 raft_consensus.cc:697] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:39.985180  3825 consensus_queue.cc:237] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:39.993721  3495 catalog_manager.cc:5697] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 0 to 1, leader changed from <none> to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 1 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.015425  3822 consensus_queue.cc:1048] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.027575  3793 consensus_queue.cc:1048] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.028023  3857 consensus_queue.cc:1048] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:40.045574  3860 consensus_queue.cc:1048] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.053593  3825 consensus_queue.cc:1048] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.058267  3796 consensus_queue.cc:1048] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.060976  3796 raft_consensus.cc:493] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.061568  3796 raft_consensus.cc:515] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.065071  3796 leader_election.cc:290] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:40.067016  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:40.068384  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:40.069617  3673 raft_consensus.cc:2393] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 915f522cefa74142aa1c5914a877dbbf in current term 1: Already voted for candidate 602b6ed4efe04787936c1f77080c61e6 in this term.
I20260709 12:22:40.072513  3714 leader_election.cc:304] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 915f522cefa74142aa1c5914a877dbbf; no voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:40.073657  3860 raft_consensus.cc:3060] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:40.085096  3860 raft_consensus.cc:2749] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:40.101436  3860 consensus_queue.cc:1048] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:40.103494  3825 consensus_queue.cc:1048] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:40.155994  3828 consensus_queue.cc:1048] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.158452  3828 consensus_queue.cc:1048] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.164656  3674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6827650806d94c81b65b6bf932270220"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
mode: GRACEFUL
new_leader_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:49760
I20260709 12:22:40.165247  3674 raft_consensus.cc:606] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Received request to transfer leadership to TS 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:40.184926  3748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
mode: GRACEFUL
new_leader_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:53760
I20260709 12:22:40.185595  3748 raft_consensus.cc:606] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Received request to transfer leadership to TS 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:40.197338  3673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
mode: GRACEFUL
new_leader_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:49760
I20260709 12:22:40.197922  3673 raft_consensus.cc:606] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Received request to transfer leadership to TS 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:40.204041  3747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
mode: GRACEFUL
new_leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:53760
I20260709 12:22:40.204783  3747 raft_consensus.cc:606] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Received request to transfer leadership to TS 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:40.220881  3598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "086558ab35914c12869f94ad34540292"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:54096
I20260709 12:22:40.221465  3598 raft_consensus.cc:606] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Received request to transfer leadership to TS 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:40.227094  3598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:54096
I20260709 12:22:40.227627  3598 raft_consensus.cc:606] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Received request to transfer leadership to TS 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:40.235744  3598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:54096
I20260709 12:22:40.236279  3598 raft_consensus.cc:606] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Received request to transfer leadership to TS 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:40.298283  3828 consensus_queue.cc:1048] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.305871  3793 raft_consensus.cc:993] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:40.306505  3829 raft_consensus.cc:1081] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Signalling peer 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:40.308624  3748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:53754
I20260709 12:22:40.309233  3748 raft_consensus.cc:493] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.309629  3748 raft_consensus.cc:3060] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.313607  3828 consensus_queue.cc:1048] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:40.316658  3748 raft_consensus.cc:515] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.319938  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:40.320549  3598 raft_consensus.cc:3055] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:40.320870  3598 raft_consensus.cc:740] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:40.321699  3598 consensus_queue.cc:260] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.322093  3748 leader_election.cc:290] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:40.322738  3598 raft_consensus.cc:3060] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.325137  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78d8d43e1af454e9be9691159fa7525" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:40.325897  3673 raft_consensus.cc:3060] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.330211  3598 raft_consensus.cc:2468] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 2.
I20260709 12:22:40.331511  3713 leader_election.cc:304] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:40.332331  3860 raft_consensus.cc:2804] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:40.332325  3673 raft_consensus.cc:2468] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 2.
I20260709 12:22:40.332934  3860 raft_consensus.cc:697] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 2 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:40.333890  3860 consensus_queue.cc:237] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.336910  3828 raft_consensus.cc:993] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:40.337481  3829 raft_consensus.cc:1081] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Signalling peer 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:40.339875  3674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "086558ab35914c12869f94ad34540292"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:22:40.340606  3674 raft_consensus.cc:493] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.340993  3674 raft_consensus.cc:3060] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.344120  3494 catalog_manager.cc:5697] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 1 to 2, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 2 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:40.348513  3674 raft_consensus.cc:515] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.350775  3674 leader_election.cc:290] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:40.351897  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:40.352299  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086558ab35914c12869f94ad34540292" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:40.352530  3598 raft_consensus.cc:3055] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:40.353076  3748 raft_consensus.cc:3060] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.353055  3598 raft_consensus.cc:740] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:40.355545  3598 consensus_queue.cc:260] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.356851  3598 raft_consensus.cc:3060] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.359448  3748 raft_consensus.cc:2468] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:40.360584  3638 leader_election.cc:304] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:40.361279  3864 raft_consensus.cc:2804] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:40.361687  3864 raft_consensus.cc:697] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:40.362519  3864 consensus_queue.cc:237] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.363504  3598 raft_consensus.cc:2468] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:40.369592  3494 catalog_manager.cc:5697] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 1 to 2, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 2 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.375398  3864 consensus_queue.cc:1048] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:40.388813  3825 consensus_queue.cc:1048] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:40.461760  3796 raft_consensus.cc:993] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf: : Instructing follower 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:40.462303  3860 raft_consensus.cc:1081] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Signalling peer 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:40.464061  3598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:54090
I20260709 12:22:40.464607  3598 raft_consensus.cc:493] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.464918  3598 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.470201  3598 raft_consensus.cc:515] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.472065  3598 leader_election.cc:290] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:40.473136  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:40.473323  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:40.473969  3674 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.474025  3748 raft_consensus.cc:3055] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:40.474692  3748 raft_consensus.cc:740] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:40.475394  3748 consensus_queue.cc:260] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.476382  3748 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.481197  3674 raft_consensus.cc:2468] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:40.481537  3748 raft_consensus.cc:2468] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:40.482877  3565 leader_election.cc:304] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:40.483734  3793 raft_consensus.cc:2804] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:40.484377  3793 raft_consensus.cc:697] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:40.485345  3793 consensus_queue.cc:237] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.492214  3497 catalog_manager.cc:5697] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 1 to 2, leader changed from 915f522cefa74142aa1c5914a877dbbf (127.23.217.67) to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.509552  3864 raft_consensus.cc:993] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c: : Instructing follower 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:40.510159  3825 raft_consensus.cc:1081] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Signalling peer 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:40.511883  3598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6827650806d94c81b65b6bf932270220"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:54088
I20260709 12:22:40.512539  3598 raft_consensus.cc:493] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.512966  3598 raft_consensus.cc:3060] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.518705  3598 raft_consensus.cc:515] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.520583  3598 leader_election.cc:290] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:40.521440  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:40.521874  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6827650806d94c81b65b6bf932270220" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:40.522156  3674 raft_consensus.cc:3055] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:40.522578  3674 raft_consensus.cc:740] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:40.522606  3748 raft_consensus.cc:3060] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.523414  3674 consensus_queue.cc:260] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.524406  3674 raft_consensus.cc:3060] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.528434  3748 raft_consensus.cc:2468] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:40.529599  3563 leader_election.cc:304] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:40.529763  3674 raft_consensus.cc:2468] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:40.530396  3793 raft_consensus.cc:2804] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:40.531024  3793 raft_consensus.cc:697] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:40.531901  3793 consensus_queue.cc:237] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.538439  3497 catalog_manager.cc:5697] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 1 to 2, leader changed from 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66) to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.580850  3864 raft_consensus.cc:993] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c: : Instructing follower 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:40.581357  3825 raft_consensus.cc:1081] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Signalling peer 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:40.582988  3598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:54088
I20260709 12:22:40.583577  3598 raft_consensus.cc:493] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.583879  3598 raft_consensus.cc:3060] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.590319  3598 raft_consensus.cc:515] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.592306  3598 leader_election.cc:290] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:40.593078  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:40.593672  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da3e8ce02cc4f2589ce772d40d87fbc" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:40.593812  3674 raft_consensus.cc:3055] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:40.594259  3674 raft_consensus.cc:740] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:40.594307  3748 raft_consensus.cc:3060] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.595389  3674 consensus_queue.cc:260] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.596686  3674 raft_consensus.cc:3060] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.602049  3748 raft_consensus.cc:2468] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:40.603329  3563 leader_election.cc:304] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:40.603569  3674 raft_consensus.cc:2468] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:40.604264  3793 raft_consensus.cc:2804] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:40.604843  3793 raft_consensus.cc:697] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:40.605762  3793 consensus_queue.cc:237] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.612854  3497 catalog_manager.cc:5697] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 1 to 2, leader changed from 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66) to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.629807  3796 raft_consensus.cc:993] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf: : Instructing follower 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:40.630355  3860 raft_consensus.cc:1081] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Signalling peer 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:40.631860  3674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:49744
I20260709 12:22:40.632428  3674 raft_consensus.cc:493] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.632746  3674 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.637681  3674 raft_consensus.cc:515] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.639281  3674 leader_election.cc:290] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:40.640336  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:40.640412  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:40.640916  3598 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.641201  3748 raft_consensus.cc:3055] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:40.641590  3748 raft_consensus.cc:740] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:40.642371  3748 consensus_queue.cc:260] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.643507  3748 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.647954  3598 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:40.649458  3639 leader_election.cc:304] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:40.649724  3748 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:40.650473  3825 raft_consensus.cc:2804] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:40.651082  3825 raft_consensus.cc:697] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:40.651911  3825 consensus_queue.cc:237] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.658453  3497 catalog_manager.cc:5697] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 1 to 2, leader changed from 915f522cefa74142aa1c5914a877dbbf (127.23.217.67) to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 2 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.679715  3828 raft_consensus.cc:993] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:40.680233  3793 raft_consensus.cc:1081] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Signalling peer 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:40.681785  3674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:22:40.682240  3674 raft_consensus.cc:493] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:40.682561  3674 raft_consensus.cc:3060] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.687582  3674 raft_consensus.cc:515] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.688995  3674 leader_election.cc:290] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:40.689917  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:40.690300  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ccf46e805348be875b04ab724ec30e" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:40.690635  3598 raft_consensus.cc:3055] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:40.690968  3748 raft_consensus.cc:3060] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.690989  3598 raft_consensus.cc:740] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:40.692006  3598 consensus_queue.cc:260] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.693279  3598 raft_consensus.cc:3060] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:40.698185  3748 raft_consensus.cc:2468] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:40.699515  3638 leader_election.cc:304] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:40.699867  3598 raft_consensus.cc:2468] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:40.700431  3825 raft_consensus.cc:2804] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:40.700984  3825 raft_consensus.cc:697] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:40.701714  3825 consensus_queue.cc:237] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:40.707896  3497 catalog_manager.cc:5697] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 1 to 2, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 2 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.749812  3598 raft_consensus.cc:1275] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.749989  3674 raft_consensus.cc:1275] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.751471  3860 consensus_queue.cc:1048] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:40.751626  3825 consensus_queue.cc:1048] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:40.762065  3598 raft_consensus.cc:1275] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.767693  3860 consensus_queue.cc:1048] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:40.768074  3748 raft_consensus.cc:1275] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.770412  3853 consensus_queue.cc:1048] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:40.980057  3674 raft_consensus.cc:1275] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.981730  3793 consensus_queue.cc:1048] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:40.985426  3748 raft_consensus.cc:1275] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.987124  3793 consensus_queue.cc:1048] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:40.990978  3747 raft_consensus.cc:1275] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.992985  3879 consensus_queue.cc:1048] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:41.002714  3674 raft_consensus.cc:1275] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.004312  3829 consensus_queue.cc:1048] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.085114  3599 raft_consensus.cc:1275] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.086915  3825 consensus_queue.cc:1048] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.095851  3748 raft_consensus.cc:1275] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.097539  3864 consensus_queue.cc:1048] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.110834  3748 raft_consensus.cc:1275] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.112250  3864 consensus_queue.cc:1048] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.120561  3599 raft_consensus.cc:1275] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.121044  3747 raft_consensus.cc:1275] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.122354  3853 consensus_queue.cc:1048] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:41.122934  3828 consensus_queue.cc:1048] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:41.136360  3674 raft_consensus.cc:1275] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:41.138006  3793 consensus_queue.cc:1048] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:43.268877  3494 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41338:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:43.271728  3494 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:43.323030  3585 tablet_service.cc:1511] Processing CreateTablet for tablet 7061fb94422f4b968a3e9364c6b7849a (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:43.323843  3663 tablet_service.cc:1511] Processing CreateTablet for tablet 7061fb94422f4b968a3e9364c6b7849a (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:43.324592  3585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7061fb94422f4b968a3e9364c6b7849a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.325202  3663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7061fb94422f4b968a3e9364c6b7849a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.326187  3732 tablet_service.cc:1511] Processing CreateTablet for tablet 7061fb94422f4b968a3e9364c6b7849a (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:43.327531  3732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7061fb94422f4b968a3e9364c6b7849a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.330397  3589 tablet_service.cc:1511] Processing CreateTablet for tablet 38b5a56d70a5451f94c458da45debd72 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:43.330905  3584 tablet_service.cc:1511] Processing CreateTablet for tablet dd23c58d49d2463ba8ebe8e135e954e5 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:43.332129  3589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b5a56d70a5451f94c458da45debd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.336476  3735 tablet_service.cc:1511] Processing CreateTablet for tablet 38b5a56d70a5451f94c458da45debd72 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:43.338573  3584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd23c58d49d2463ba8ebe8e135e954e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.338006  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b5a56d70a5451f94c458da45debd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.339267  3657 tablet_service.cc:1511] Processing CreateTablet for tablet 38b5a56d70a5451f94c458da45debd72 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:43.340576  3657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b5a56d70a5451f94c458da45debd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.344812  3734 tablet_service.cc:1511] Processing CreateTablet for tablet dd23c58d49d2463ba8ebe8e135e954e5 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:43.345434  3586 tablet_service.cc:1511] Processing CreateTablet for tablet 6052b004c40f4059a30d727097987503 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:43.347334  3586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6052b004c40f4059a30d727097987503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.348065  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd23c58d49d2463ba8ebe8e135e954e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.347235  3662 tablet_service.cc:1511] Processing CreateTablet for tablet dd23c58d49d2463ba8ebe8e135e954e5 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:43.350770  3662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd23c58d49d2463ba8ebe8e135e954e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.358309  3664 tablet_service.cc:1511] Processing CreateTablet for tablet 6052b004c40f4059a30d727097987503 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:43.360174  3664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6052b004c40f4059a30d727097987503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.377199  3733 tablet_service.cc:1511] Processing CreateTablet for tablet 6052b004c40f4059a30d727097987503 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:43.378674  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6052b004c40f4059a30d727097987503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.379925  3588 tablet_service.cc:1511] Processing CreateTablet for tablet 6ab5aba566ee42828060e916fd6b3173 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:43.381490  3588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab5aba566ee42828060e916fd6b3173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.383554  3660 tablet_service.cc:1511] Processing CreateTablet for tablet 6ab5aba566ee42828060e916fd6b3173 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:43.384919  3660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab5aba566ee42828060e916fd6b3173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.385146  3738 tablet_service.cc:1511] Processing CreateTablet for tablet 6ab5aba566ee42828060e916fd6b3173 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:43.386343  3738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab5aba566ee42828060e916fd6b3173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.399165  3587 tablet_service.cc:1511] Processing CreateTablet for tablet 25a07ac21d3946dca3d7416afaee7243 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:43.400599  3587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a07ac21d3946dca3d7416afaee7243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.407317  3661 tablet_service.cc:1511] Processing CreateTablet for tablet 25a07ac21d3946dca3d7416afaee7243 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:43.410017  3661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a07ac21d3946dca3d7416afaee7243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.415596  3737 tablet_service.cc:1511] Processing CreateTablet for tablet 25a07ac21d3946dca3d7416afaee7243 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:43.416957  3737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a07ac21d3946dca3d7416afaee7243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.420416  3897 tablet_bootstrap.cc:492] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:43.423029  3583 tablet_service.cc:1511] Processing CreateTablet for tablet 5198ba68cd8342098f9b5e712b0bf1fc (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:43.424650  3583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5198ba68cd8342098f9b5e712b0bf1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.429332  3659 tablet_service.cc:1511] Processing CreateTablet for tablet 5198ba68cd8342098f9b5e712b0bf1fc (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:43.435012  3659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5198ba68cd8342098f9b5e712b0bf1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.442096  3732 tablet_service.cc:1511] Processing CreateTablet for tablet 5198ba68cd8342098f9b5e712b0bf1fc (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:43.448498  3732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5198ba68cd8342098f9b5e712b0bf1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.466079  3898 tablet_bootstrap.cc:492] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:43.467638  3586 tablet_service.cc:1511] Processing CreateTablet for tablet 4f61e3b9e450427f856143ce283b4eda (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:43.477895  3897 tablet_bootstrap.cc:654] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.491475  3899 tablet_bootstrap.cc:492] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:43.491941  3586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f61e3b9e450427f856143ce283b4eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.483994  3657 tablet_service.cc:1511] Processing CreateTablet for tablet 4f61e3b9e450427f856143ce283b4eda (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:43.495271  3657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f61e3b9e450427f856143ce283b4eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.485591  3738 tablet_service.cc:1511] Processing CreateTablet for tablet 4f61e3b9e450427f856143ce283b4eda (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:43.487356  3588 tablet_service.cc:1511] Processing CreateTablet for tablet 935880ce84e04dd1aa02e28cebb1df50 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:43.488435  3735 tablet_service.cc:1511] Processing CreateTablet for tablet 935880ce84e04dd1aa02e28cebb1df50 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:43.489265  3660 tablet_service.cc:1511] Processing CreateTablet for tablet 935880ce84e04dd1aa02e28cebb1df50 (DEFAULT_TABLE table=soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:43.499312  3588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 935880ce84e04dd1aa02e28cebb1df50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.499837  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 935880ce84e04dd1aa02e28cebb1df50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.500181  3660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 935880ce84e04dd1aa02e28cebb1df50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.498243  3897 tablet_bootstrap.cc:492] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:43.502969  3897 ts_tablet_manager.cc:1403] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.083s	user 0.006s	sys 0.009s
I20260709 12:22:43.503197  3738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f61e3b9e450427f856143ce283b4eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:43.506873  3897 raft_consensus.cc:359] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.507834  3897 raft_consensus.cc:385] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.508196  3897 raft_consensus.cc:740] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.515745  3898 tablet_bootstrap.cc:654] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.516573  3897 consensus_queue.cc:260] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.530376  3897 ts_tablet_manager.cc:1434] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.027s	user 0.001s	sys 0.007s
I20260709 12:22:43.533182  3897 tablet_bootstrap.cc:492] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:43.530372  3899 tablet_bootstrap.cc:654] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.549839  3897 tablet_bootstrap.cc:654] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.557510  3899 tablet_bootstrap.cc:492] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:43.558303  3899 ts_tablet_manager.cc:1403] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.000s
I20260709 12:22:43.575389  3899 raft_consensus.cc:359] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.576325  3899 raft_consensus.cc:385] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.576691  3899 raft_consensus.cc:740] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.577649  3899 consensus_queue.cc:260] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.580235  3899 ts_tablet_manager.cc:1434] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:22:43.581352  3899 tablet_bootstrap.cc:492] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:43.583717  3898 tablet_bootstrap.cc:492] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:43.584548  3898 ts_tablet_manager.cc:1403] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.119s	user 0.008s	sys 0.021s
I20260709 12:22:43.589170  3899 tablet_bootstrap.cc:654] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.588621  3898 raft_consensus.cc:359] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.595022  3898 raft_consensus.cc:385] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.595453  3898 raft_consensus.cc:740] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.596451  3898 consensus_queue.cc:260] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.599620  3898 ts_tablet_manager.cc:1434] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.004s
I20260709 12:22:43.601037  3898 tablet_bootstrap.cc:492] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:43.609773  3898 tablet_bootstrap.cc:654] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.627332  3898 tablet_bootstrap.cc:492] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:43.628005  3898 ts_tablet_manager.cc:1403] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.011s
I20260709 12:22:43.633512  3898 raft_consensus.cc:359] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.634274  3898 raft_consensus.cc:385] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.634634  3898 raft_consensus.cc:740] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.635044  3897 tablet_bootstrap.cc:492] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:43.635627  3897 ts_tablet_manager.cc:1403] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.103s	user 0.034s	sys 0.048s
I20260709 12:22:43.635506  3898 consensus_queue.cc:260] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.636065  3899 tablet_bootstrap.cc:492] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:43.636754  3899 ts_tablet_manager.cc:1403] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.056s	user 0.028s	sys 0.022s
I20260709 12:22:43.637919  3898 ts_tablet_manager.cc:1434] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:22:43.638805  3897 raft_consensus.cc:359] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.639110  3898 tablet_bootstrap.cc:492] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:43.639549  3897 raft_consensus.cc:385] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.644443  3897 raft_consensus.cc:740] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.645228  3897 consensus_queue.cc:260] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.647225  3898 tablet_bootstrap.cc:654] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.639876  3899 raft_consensus.cc:359] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.649761  3899 raft_consensus.cc:385] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.650128  3899 raft_consensus.cc:740] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.651033  3899 consensus_queue.cc:260] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.653270  3899 ts_tablet_manager.cc:1434] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:22:43.654367  3899 tablet_bootstrap.cc:492] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:43.657069  3897 ts_tablet_manager.cc:1434] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:22:43.658895  3898 tablet_bootstrap.cc:492] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:43.659411  3898 ts_tablet_manager.cc:1403] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.010s
I20260709 12:22:43.661603  3899 tablet_bootstrap.cc:654] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.662461  3898 raft_consensus.cc:359] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.663291  3898 raft_consensus.cc:385] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.663587  3898 raft_consensus.cc:740] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.664487  3898 consensus_queue.cc:260] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.665037  3897 tablet_bootstrap.cc:492] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:43.666973  3898 ts_tablet_manager.cc:1434] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:22:43.669296  3898 tablet_bootstrap.cc:492] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:43.671051  3885 raft_consensus.cc:493] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.671815  3885 raft_consensus.cc:515] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.672441  3897 tablet_bootstrap.cc:654] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.676064  3898 tablet_bootstrap.cc:654] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.676045  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:43.677350  3599 raft_consensus.cc:2468] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 0.
I20260709 12:22:43.683959  3713 leader_election.cc:304] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:43.689155  3898 tablet_bootstrap.cc:492] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:43.689730  3898 ts_tablet_manager.cc:1403] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 12:22:43.693441  3898 raft_consensus.cc:359] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.694413  3898 raft_consensus.cc:385] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.694813  3898 raft_consensus.cc:740] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.695819  3898 consensus_queue.cc:260] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.697711  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:43.698354  3898 ts_tablet_manager.cc:1434] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:22:43.712316  3898 tablet_bootstrap.cc:492] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
W20260709 12:22:43.712803  3714 leader_election.cc:343] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:43.698702  3885 leader_election.cc:290] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:43.724643  3885 raft_consensus.cc:2804] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:43.724969  3898 tablet_bootstrap.cc:654] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.725343  3885 raft_consensus.cc:493] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.733618  3885 raft_consensus.cc:3060] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.743458  3898 tablet_bootstrap.cc:492] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:43.744232  3898 ts_tablet_manager.cc:1403] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.006s
I20260709 12:22:43.748049  3898 raft_consensus.cc:359] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.748986  3898 raft_consensus.cc:385] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.749361  3898 raft_consensus.cc:740] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.750278  3898 consensus_queue.cc:260] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.753278  3898 ts_tablet_manager.cc:1434] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:22:43.754670  3898 tablet_bootstrap.cc:492] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:43.762590  3898 tablet_bootstrap.cc:654] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:43.766343  3713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.65:43629, user_credentials={real_user=slave}} blocked reactor thread for 82525.8us
W20260709 12:22:43.774899  3565 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:40139, user_credentials={real_user=slave}} blocked reactor thread for 72396.3us
I20260709 12:22:43.778079  3898 tablet_bootstrap.cc:492] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:43.779099  3898 ts_tablet_manager.cc:1403] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.017s
I20260709 12:22:43.783254  3898 raft_consensus.cc:359] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.784343  3898 raft_consensus.cc:385] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.784802  3898 raft_consensus.cc:740] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.785864  3898 consensus_queue.cc:260] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.788866  3898 ts_tablet_manager.cc:1434] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:43.790083  3898 tablet_bootstrap.cc:492] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:43.790668  3899 tablet_bootstrap.cc:492] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:43.789885  3885 raft_consensus.cc:515] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.794266  3899 ts_tablet_manager.cc:1403] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.140s	user 0.059s	sys 0.048s
I20260709 12:22:43.794443  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:43.795171  3599 raft_consensus.cc:3060] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.797855  3899 raft_consensus.cc:359] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.798652  3899 raft_consensus.cc:385] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.798936  3899 raft_consensus.cc:740] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.799742  3899 consensus_queue.cc:260] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.802100  3898 tablet_bootstrap.cc:654] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.802217  3899 ts_tablet_manager.cc:1434] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.002s
I20260709 12:22:43.803665  3899 tablet_bootstrap.cc:492] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:43.804363  3793 raft_consensus.cc:493] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.804920  3793 raft_consensus.cc:515] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.808876  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:43.810277  3897 tablet_bootstrap.cc:492] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:43.810992  3899 tablet_bootstrap.cc:654] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.810848  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:43.810902  3897 ts_tablet_manager.cc:1403] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.146s	user 0.056s	sys 0.033s
W20260709 12:22:43.812664  3565 leader_election.cc:343] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:43.811609  3748 raft_consensus.cc:2468] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:43.815085  3897 raft_consensus.cc:359] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.815608  3563 leader_election.cc:304] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:43.816061  3897 raft_consensus.cc:385] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.816535  3897 raft_consensus.cc:740] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.816728  3913 raft_consensus.cc:2804] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:43.817281  3913 raft_consensus.cc:493] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.817462  3897 consensus_queue.cc:260] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.813656  3793 leader_election.cc:290] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:43.817613  3913 raft_consensus.cc:3060] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.824582  3897 ts_tablet_manager.cc:1434] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:22:43.825647  3897 tablet_bootstrap.cc:492] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:43.838133  3897 tablet_bootstrap.cc:654] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.840149  3899 tablet_bootstrap.cc:492] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:43.840951  3899 ts_tablet_manager.cc:1403] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.010s
I20260709 12:22:43.842154  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
W20260709 12:22:43.843847  3714 leader_election.cc:343] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:43.844820  3885 leader_election.cc:290] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:43.844556  3899 raft_consensus.cc:359] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.845392  3899 raft_consensus.cc:385] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.845535  3599 raft_consensus.cc:2468] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 1.
I20260709 12:22:43.847246  3713 leader_election.cc:304] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:43.848834  3885 raft_consensus.cc:2804] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:43.845741  3899 raft_consensus.cc:740] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.850337  3885 raft_consensus.cc:697] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:43.851166  3899 consensus_queue.cc:260] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.851397  3897 tablet_bootstrap.cc:492] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:43.852517  3897 ts_tablet_manager.cc:1403] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.006s
I20260709 12:22:43.853819  3899 ts_tablet_manager.cc:1434] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:22:43.853668  3913 raft_consensus.cc:515] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.856954  3901 raft_consensus.cc:493] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.856689  3897 raft_consensus.cc:359] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.857502  3897 raft_consensus.cc:385] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.857450  3901 raft_consensus.cc:515] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.857607  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:43.857963  3897 raft_consensus.cc:740] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.858646  3673 raft_consensus.cc:3060] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.858920  3899 tablet_bootstrap.cc:492] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:43.859767  3901 leader_election.cc:290] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:43.858990  3897 consensus_queue.cc:260] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.863193  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6052b004c40f4059a30d727097987503" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:43.865824  3897 ts_tablet_manager.cc:1434] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
W20260709 12:22:43.866827  3638 leader_election.cc:343] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:43.866989  3897 tablet_bootstrap.cc:492] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:43.867137  3899 tablet_bootstrap.cc:654] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.874531  3897 tablet_bootstrap.cc:654] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.886018  3897 tablet_bootstrap.cc:492] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:43.886624  3897 ts_tablet_manager.cc:1403] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.003s
I20260709 12:22:43.887362  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6052b004c40f4059a30d727097987503" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:43.851552  3885 consensus_queue.cc:237] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.889213  3911 raft_consensus.cc:493] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.890519  3897 raft_consensus.cc:359] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.891513  3911 raft_consensus.cc:515] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.892973  3899 tablet_bootstrap.cc:492] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:43.899310  3899 ts_tablet_manager.cc:1403] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.000s
I20260709 12:22:43.899485  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:43.900297  3598 raft_consensus.cc:2468] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 0.
I20260709 12:22:43.902586  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:43.902894  3899 raft_consensus.cc:359] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.903285  3674 raft_consensus.cc:2468] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 0.
I20260709 12:22:43.903662  3899 raft_consensus.cc:385] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.903985  3899 raft_consensus.cc:740] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.904662  3713 leader_election.cc:304] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:43.904829  3899 consensus_queue.cc:260] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.904175  3911 leader_election.cc:290] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:43.907922  3921 raft_consensus.cc:2804] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:43.890842  3599 raft_consensus.cc:2468] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:43.908655  3921 raft_consensus.cc:493] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.915973  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:43.917639  3921 raft_consensus.cc:3060] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.917852  3748 raft_consensus.cc:3060] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.928555  3921 raft_consensus.cc:515] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.933424  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:43.935990  3921 leader_election.cc:290] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:43.938400  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:43.939409  3674 raft_consensus.cc:3060] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.934381  3598 raft_consensus.cc:3060] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.957839  3913 leader_election.cc:290] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:43.958782  3639 leader_election.cc:304] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:43.960862  3901 raft_consensus.cc:2804] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:43.961410  3901 raft_consensus.cc:493] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.961827  3901 raft_consensus.cc:3060] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:43.891520  3897 raft_consensus.cc:385] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.964224  3897 raft_consensus.cc:740] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.965260  3898 tablet_bootstrap.cc:492] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:43.964985  3897 consensus_queue.cc:260] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.965783  3898 ts_tablet_manager.cc:1403] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.176s	user 0.032s	sys 0.077s
I20260709 12:22:43.967366  3897 ts_tablet_manager.cc:1434] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 12:22:43.968395  3897 tablet_bootstrap.cc:492] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:43.894315  3673 raft_consensus.cc:2468] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:43.976264  3897 tablet_bootstrap.cc:654] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:43.976392  3565 leader_election.cc:304] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:43.981559  3674 raft_consensus.cc:2468] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 1.
I20260709 12:22:43.982366  3913 raft_consensus.cc:2804] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:43.982832  3714 leader_election.cc:304] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:43.983131  3748 raft_consensus.cc:2468] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:43.985316  3897 tablet_bootstrap.cc:492] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:43.985924  3897 ts_tablet_manager.cc:1403] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.003s
I20260709 12:22:43.987627  3898 raft_consensus.cc:359] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.988300  3899 ts_tablet_manager.cc:1434] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.089s	user 0.029s	sys 0.050s
I20260709 12:22:43.988544  3898 raft_consensus.cc:385] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:43.988220  3922 raft_consensus.cc:2804] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:43.988967  3898 raft_consensus.cc:740] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:43.989576  3899 tablet_bootstrap.cc:492] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:43.991505  3898 consensus_queue.cc:260] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.993038  3853 raft_consensus.cc:493] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.994109  3853 raft_consensus.cc:515] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.994729  3898 ts_tablet_manager.cc:1434] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 12:22:43.993899  3922 raft_consensus.cc:697] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:43.996106  3898 tablet_bootstrap.cc:492] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:43.995853  3922 consensus_queue.cc:237] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:43.997277  3853 leader_election.cc:290] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:43.997447  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a07ac21d3946dca3d7416afaee7243" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:43.990199  3913 raft_consensus.cc:697] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
W20260709 12:22:43.999523  3638 leader_election.cc:343] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:44.003320  3793 raft_consensus.cc:493] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.004086  3793 raft_consensus.cc:515] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.005954  3913 consensus_queue.cc:237] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.006945  3921 raft_consensus.cc:493] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.007551  3921 raft_consensus.cc:515] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.008236  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:44.008966  3674 raft_consensus.cc:2468] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:44.009099  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:44.009703  3748 raft_consensus.cc:2468] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:44.008116  3793 leader_election.cc:290] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.010071  3565 leader_election.cc:304] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:44.010618  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:44.011221  3793 raft_consensus.cc:2804] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:44.011812  3921 leader_election.cc:290] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:44.012024  3793 raft_consensus.cc:493] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.012719  3793 raft_consensus.cc:3060] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.012809  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:43.997493  3897 raft_consensus.cc:359] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.013594  3674 raft_consensus.cc:2468] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 0.
I20260709 12:22:44.013790  3897 raft_consensus.cc:385] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.014411  3897 raft_consensus.cc:740] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.014920  3714 leader_election.cc:304] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.015928  3921 raft_consensus.cc:2804] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:44.015460  3897 consensus_queue.cc:260] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.016415  3921 raft_consensus.cc:493] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.016944  3921 raft_consensus.cc:3060] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.022555  3901 raft_consensus.cc:515] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.021683  3793 raft_consensus.cc:515] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.026325  3899 tablet_bootstrap.cc:654] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.029202  3599 raft_consensus.cc:2393] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 915f522cefa74142aa1c5914a877dbbf in current term 1: Already voted for candidate 602b6ed4efe04787936c1f77080c61e6 in this term.
I20260709 12:22:44.027226  3897 ts_tablet_manager.cc:1434] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.007s
I20260709 12:22:44.020200  3598 raft_consensus.cc:2468] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 1.
I20260709 12:22:44.031262  3597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a07ac21d3946dca3d7416afaee7243" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:44.033674  3597 raft_consensus.cc:2468] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:44.032776  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6052b004c40f4059a30d727097987503" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:44.040092  3596 raft_consensus.cc:3060] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.040829  3921 raft_consensus.cc:515] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.042989  3901 leader_election.cc:290] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.035557  3897 tablet_bootstrap.cc:492] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:44.036223  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6052b004c40f4059a30d727097987503" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.044730  3748 raft_consensus.cc:3060] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.037985  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:44.038475  3793 leader_election.cc:290] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.038751  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.046622  3674 raft_consensus.cc:3060] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.047106  3639 leader_election.cc:304] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:44.049849  3747 raft_consensus.cc:2393] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 602b6ed4efe04787936c1f77080c61e6 in current term 1: Already voted for candidate 915f522cefa74142aa1c5914a877dbbf in this term.
I20260709 12:22:44.055259  3898 tablet_bootstrap.cc:654] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.055979  3495 catalog_manager.cc:5697] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.065167  3748 raft_consensus.cc:2468] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:44.055980  3925 raft_consensus.cc:2804] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:44.049693  3496 catalog_manager.cc:5697] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 1 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:44.059816  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:44.060123  3921 leader_election.cc:290] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:44.053155  3674 raft_consensus.cc:2468] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:44.066857  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:44.079226  3673 raft_consensus.cc:2393] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 915f522cefa74142aa1c5914a877dbbf in current term 1: Already voted for candidate 602b6ed4efe04787936c1f77080c61e6 in this term.
I20260709 12:22:44.071086  3638 leader_election.cc:304] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.080678  3897 tablet_bootstrap.cc:654] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.081140  3901 raft_consensus.cc:2804] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.081761  3901 raft_consensus.cc:697] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:44.081902  3599 raft_consensus.cc:2393] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 915f522cefa74142aa1c5914a877dbbf in current term 1: Already voted for candidate 602b6ed4efe04787936c1f77080c61e6 in this term.
I20260709 12:22:44.075631  3925 raft_consensus.cc:493] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.082789  3901 consensus_queue.cc:237] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.083408  3925 raft_consensus.cc:3060] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.084990  3565 leader_election.cc:304] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:44.074791  3596 raft_consensus.cc:2468] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:44.092758  3714 leader_election.cc:304] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 915f522cefa74142aa1c5914a877dbbf; no voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:44.096820  3913 raft_consensus.cc:2804] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.097728  3922 raft_consensus.cc:2749] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:44.098527  3913 raft_consensus.cc:697] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.099766  3913 consensus_queue.cc:237] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.104106  3897 tablet_bootstrap.cc:492] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:44.104892  3897 ts_tablet_manager.cc:1403] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.008s
I20260709 12:22:44.109022  3897 raft_consensus.cc:359] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.109113  3925 raft_consensus.cc:515] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.109764  3897 raft_consensus.cc:385] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.110164  3897 raft_consensus.cc:740] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.120625  3899 tablet_bootstrap.cc:492] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:44.119429  3897 consensus_queue.cc:260] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.121660  3898 tablet_bootstrap.cc:492] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:44.121320  3899 ts_tablet_manager.cc:1403] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.132s	user 0.030s	sys 0.021s
I20260709 12:22:44.123010  3898 ts_tablet_manager.cc:1403] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.127s	user 0.011s	sys 0.022s
I20260709 12:22:44.121264  3925 leader_election.cc:290] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.125021  3897 ts_tablet_manager.cc:1434] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 12:22:44.126159  3897 tablet_bootstrap.cc:492] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:44.126339  3899 raft_consensus.cc:359] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.126607  3898 raft_consensus.cc:359] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.127341  3899 raft_consensus.cc:385] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.127429  3898 raft_consensus.cc:385] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.127856  3899 raft_consensus.cc:740] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.127945  3898 raft_consensus.cc:740] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.128783  3898 consensus_queue.cc:260] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.128748  3899 consensus_queue.cc:260] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.131357  3898 ts_tablet_manager.cc:1434] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:22:44.132336  3899 ts_tablet_manager.cc:1434] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:22:44.132526  3898 tablet_bootstrap.cc:492] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: Bootstrap starting.
I20260709 12:22:44.133529  3899 tablet_bootstrap.cc:492] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:44.176812  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a07ac21d3946dca3d7416afaee7243" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:44.180007  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a07ac21d3946dca3d7416afaee7243" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.183010  3901 raft_consensus.cc:493] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.187561  3901 raft_consensus.cc:515] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.195216  3596 raft_consensus.cc:3060] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:44.196843  3638 leader_election.cc:343] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:44.205173  3901 leader_election.cc:290] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.209668  3899 tablet_bootstrap.cc:654] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.213539  3897 tablet_bootstrap.cc:654] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.216634  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:44.220472  3793 raft_consensus.cc:493] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.221748  3793 raft_consensus.cc:515] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.223137  3599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:44.224233  3599 raft_consensus.cc:2393] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522b9a515e00491e9c1dbc9a27ac099c in current term 1: Already voted for candidate 915f522cefa74142aa1c5914a877dbbf in this term.
I20260709 12:22:44.228627  3639 leader_election.cc:304] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c; no voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:44.230396  3925 raft_consensus.cc:3060] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.231086  3793 leader_election.cc:290] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.232213  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
W20260709 12:22:44.233999  3565 leader_election.cc:343] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:44.238749  3596 raft_consensus.cc:2468] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:44.243510  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:44.249042  3639 leader_election.cc:304] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 915f522cefa74142aa1c5914a877dbbf
W20260709 12:22:44.250998  3563 leader_election.cc:343] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:44.251536  3563 leader_election.cc:304] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6; no voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:44.253942  3925 raft_consensus.cc:2749] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:44.255443  3793 raft_consensus.cc:2749] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:44.256429  3853 raft_consensus.cc:2804] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.257131  3853 raft_consensus.cc:697] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:44.258051  3898 tablet_bootstrap.cc:654] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.258651  3853 consensus_queue.cc:237] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
W20260709 12:22:44.260274  3545 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:44.265468  3494 catalog_manager.cc:5697] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.268887  3899 tablet_bootstrap.cc:492] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:44.270229  3899 ts_tablet_manager.cc:1403] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.008s
I20260709 12:22:44.270437  3897 tablet_bootstrap.cc:492] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:44.272047  3897 ts_tablet_manager.cc:1403] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.146s	user 0.024s	sys 0.022s
I20260709 12:22:44.284844  3897 raft_consensus.cc:359] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.286180  3897 raft_consensus.cc:385] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.286558  3898 tablet_bootstrap.cc:492] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: No bootstrap required, opened a new log
I20260709 12:22:44.286667  3897 raft_consensus.cc:740] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.287557  3898 ts_tablet_manager.cc:1403] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: Time spent bootstrapping tablet: real 0.155s	user 0.017s	sys 0.006s
I20260709 12:22:44.279129  3899 raft_consensus.cc:359] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.288389  3897 consensus_queue.cc:260] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.288439  3899 raft_consensus.cc:385] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.293192  3899 raft_consensus.cc:740] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.295050  3897 ts_tablet_manager.cc:1434] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.023s	user 0.012s	sys 0.001s
I20260709 12:22:44.294265  3496 catalog_manager.cc:5697] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 1 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:44.296304  3898 raft_consensus.cc:359] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.297225  3897 tablet_bootstrap.cc:492] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: Bootstrap starting.
I20260709 12:22:44.297425  3898 raft_consensus.cc:385] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.297802  3898 raft_consensus.cc:740] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.298528  3899 consensus_queue.cc:260] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.302259  3899 ts_tablet_manager.cc:1434] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:22:44.305884  3495 catalog_manager.cc:5697] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 0 to 1, leader changed from <none> to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 1 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.307173  3897 tablet_bootstrap.cc:654] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.308017  3899 tablet_bootstrap.cc:492] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:44.305105  3898 consensus_queue.cc:260] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.314895  3898 ts_tablet_manager.cc:1434] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.003s
I20260709 12:22:44.318269  3899 tablet_bootstrap.cc:654] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.325027  3897 tablet_bootstrap.cc:492] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: No bootstrap required, opened a new log
I20260709 12:22:44.325662  3897 ts_tablet_manager.cc:1403] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.013s
I20260709 12:22:44.328550  3495 catalog_manager.cc:5697] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 0 to 1, leader changed from <none> to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 1 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.329622  3897 raft_consensus.cc:359] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.331218  3793 raft_consensus.cc:493] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.331532  3897 raft_consensus.cc:385] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.332056  3897 raft_consensus.cc:740] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.331885  3793 raft_consensus.cc:515] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.333246  3897 consensus_queue.cc:260] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.335295  3897 ts_tablet_manager.cc:1434] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 12:22:44.335614  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:44.336431  3673 raft_consensus.cc:2468] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:44.338025  3565 leader_election.cc:304] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:44.339214  3913 raft_consensus.cc:2804] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:44.339628  3913 raft_consensus.cc:493] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.340021  3913 raft_consensus.cc:3060] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.340723  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:44.341523  3899 tablet_bootstrap.cc:492] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:44.341524  3747 raft_consensus.cc:2468] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:44.342471  3899 ts_tablet_manager.cc:1403] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.010s
I20260709 12:22:44.341619  3793 leader_election.cc:290] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.347061  3899 raft_consensus.cc:359] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.347879  3899 raft_consensus.cc:385] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.348153  3899 raft_consensus.cc:740] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.348999  3899 consensus_queue.cc:260] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.351790  3899 ts_tablet_manager.cc:1434] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:44.353057  3899 tablet_bootstrap.cc:492] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf: Bootstrap starting.
I20260709 12:22:44.355594  3913 raft_consensus.cc:515] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.358862  3913 leader_election.cc:290] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.359591  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:44.360322  3673 raft_consensus.cc:3060] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.360059  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.360914  3899 tablet_bootstrap.cc:654] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.360932  3747 raft_consensus.cc:3060] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.367496  3899 tablet_bootstrap.cc:492] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf: No bootstrap required, opened a new log
I20260709 12:22:44.368057  3899 ts_tablet_manager.cc:1403] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:22:44.368094  3673 raft_consensus.cc:2468] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:44.369467  3565 leader_election.cc:304] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:44.370687  3913 raft_consensus.cc:2804] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.371637  3899 raft_consensus.cc:359] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.372102  3913 raft_consensus.cc:697] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.372440  3899 raft_consensus.cc:385] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.372733  3899 raft_consensus.cc:740] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.373580  3899 consensus_queue.cc:260] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.375272  3913 consensus_queue.cc:237] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.376478  3899 ts_tablet_manager.cc:1434] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:22:44.378965  3922 raft_consensus.cc:493] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.379503  3922 raft_consensus.cc:515] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.380590  3747 raft_consensus.cc:2468] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:44.381258  3793 raft_consensus.cc:493] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.381803  3793 raft_consensus.cc:515] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.382234  3922 leader_election.cc:290] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:44.384011  3793 leader_election.cc:290] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.386147  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:44.386425  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a07ac21d3946dca3d7416afaee7243" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:44.386974  3748 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:44.387508  3596 raft_consensus.cc:2393] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 915f522cefa74142aa1c5914a877dbbf in current term 1: Already voted for candidate 522b9a515e00491e9c1dbc9a27ac099c in this term.
I20260709 12:22:44.388137  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a07ac21d3946dca3d7416afaee7243" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:44.389343  3563 leader_election.cc:304] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.389451  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c" is_pre_election: true
I20260709 12:22:44.390278  3674 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 0.
I20260709 12:22:44.391358  3912 raft_consensus.cc:2804] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:44.391841  3912 raft_consensus.cc:493] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.392204  3912 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.392989  3713 leader_election.cc:304] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 915f522cefa74142aa1c5914a877dbbf; no voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:44.393810  3922 raft_consensus.cc:3060] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.396586  3921 consensus_queue.cc:1048] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:44.403631  3793 consensus_queue.cc:1048] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.406561  3922 raft_consensus.cc:2749] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:44.423542  3495 catalog_manager.cc:5697] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.426689  3885 consensus_queue.cc:1048] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:44.429849  3945 consensus_queue.cc:1048] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:44.433261  3853 raft_consensus.cc:493] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.433889  3853 raft_consensus.cc:515] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.436563  3853 leader_election.cc:290] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.438324  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6" is_pre_election: true
I20260709 12:22:44.439342  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf" is_pre_election: true
I20260709 12:22:44.441332  3747 raft_consensus.cc:2468] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:44.439360  3596 raft_consensus.cc:2468] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 0.
I20260709 12:22:44.443114  3638 leader_election.cc:304] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.444743  3853 raft_consensus.cc:2804] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:44.445293  3853 raft_consensus.cc:493] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.445765  3853 raft_consensus.cc:3060] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.433009  3912 raft_consensus.cc:515] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.457194  3912 leader_election.cc:290] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.458088  3672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:44.458580  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.459076  3672 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.459400  3747 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.461578  3853 raft_consensus.cc:515] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.465060  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:44.465780  3596 raft_consensus.cc:3060] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.465917  3853 leader_election.cc:290] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.466323  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.467018  3748 raft_consensus.cc:3060] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.472704  3747 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:44.474582  3563 leader_election.cc:304] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.475208  3596 raft_consensus.cc:2468] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:44.475618  3912 raft_consensus.cc:2804] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.476202  3748 raft_consensus.cc:2468] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 1.
I20260709 12:22:44.476696  3912 raft_consensus.cc:697] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.477458  3638 leader_election.cc:304] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.478544  3853 raft_consensus.cc:2804] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.478375  3912 consensus_queue.cc:237] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.479382  3853 raft_consensus.cc:697] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:44.480291  3853 consensus_queue.cc:237] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.480844  3672 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 1.
I20260709 12:22:44.503079  3495 catalog_manager.cc:5697] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 1 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.508805  3496 catalog_manager.cc:5697] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 0 to 1, leader changed from <none> to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 1 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.517596  3885 consensus_queue.cc:1048] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.534041  3885 consensus_queue.cc:1048] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:44.564363  3912 consensus_queue.cc:1048] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.576062  3913 consensus_queue.cc:1048] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.597150  3747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
mode: GRACEFUL
new_leader_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:53760
I20260709 12:22:44.598116  3747 raft_consensus.cc:606] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Received request to transfer leadership to TS 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:44.606750  3747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
mode: GRACEFUL
new_leader_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:53760
I20260709 12:22:44.607391  3747 raft_consensus.cc:606] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Received request to transfer leadership to TS 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:44.612524  3674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6052b004c40f4059a30d727097987503"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
mode: GRACEFUL
new_leader_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:49760
I20260709 12:22:44.613162  3674 raft_consensus.cc:606] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Received request to transfer leadership to TS 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:44.619526  3596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:54096
I20260709 12:22:44.620275  3596 raft_consensus.cc:606] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Received request to transfer leadership to TS 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:44.622084  3853 consensus_queue.cc:1048] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.628911  3596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:54096
I20260709 12:22:44.629403  3596 raft_consensus.cc:606] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Received request to transfer leadership to TS 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:44.638062  3596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:54096
I20260709 12:22:44.638203  3853 consensus_queue.cc:1048] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:44.638731  3596 raft_consensus.cc:606] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Received request to transfer leadership to TS 522b9a515e00491e9c1dbc9a27ac099c
I20260709 12:22:44.647675  3901 raft_consensus.cc:993] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: : Instructing follower 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:44.648197  3853 raft_consensus.cc:1081] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Signalling peer 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:44.649824  3674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
mode: GRACEFUL
new_leader_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:49760
I20260709 12:22:44.650458  3674 raft_consensus.cc:606] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Received request to transfer leadership to TS 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:44.651439  3596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6052b004c40f4059a30d727097987503"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:54088
I20260709 12:22:44.652237  3596 raft_consensus.cc:493] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:44.652750  3596 raft_consensus.cc:3060] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.659265  3596 raft_consensus.cc:515] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.661666  3596 leader_election.cc:290] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.661957  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6052b004c40f4059a30d727097987503" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:44.662421  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6052b004c40f4059a30d727097987503" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.662652  3674 raft_consensus.cc:3055] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:44.663152  3674 raft_consensus.cc:740] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:44.663156  3747 raft_consensus.cc:3060] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.664181  3674 consensus_queue.cc:260] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.665474  3674 raft_consensus.cc:3060] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.670853  3747 raft_consensus.cc:2468] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:44.671646  3674 raft_consensus.cc:2468] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:44.672066  3563 leader_election.cc:304] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.672982  3913 raft_consensus.cc:2804] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:44.673492  3913 raft_consensus.cc:697] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.674391  3913 consensus_queue.cc:237] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.681049  3495 catalog_manager.cc:5697] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 1 to 2, leader changed from 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66) to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.756247  3853 consensus_queue.cc:1048] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.769537  3853 consensus_queue.cc:1048] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.791054  3913 consensus_queue.cc:1048] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.799221  3953 raft_consensus.cc:993] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:44.799711  3953 raft_consensus.cc:1081] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Signalling peer 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:44.802196  3672 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:22:44.803184  3672 raft_consensus.cc:493] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:44.803574  3672 raft_consensus.cc:3060] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.808750  3953 consensus_queue.cc:1048] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:44.810635  3672 raft_consensus.cc:515] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.814221  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:44.814429  3748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5198ba68cd8342098f9b5e712b0bf1fc" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.814921  3596 raft_consensus.cc:3055] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:44.815390  3596 raft_consensus.cc:740] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.817567  3596 consensus_queue.cc:260] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.818619  3596 raft_consensus.cc:3060] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.819080  3672 leader_election.cc:290] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.828212  3596 raft_consensus.cc:2468] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:44.829718  3639 leader_election.cc:304] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:44.830667  3853 raft_consensus.cc:2804] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:44.831121  3853 raft_consensus.cc:697] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:44.831995  3853 consensus_queue.cc:237] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.838712  3885 raft_consensus.cc:993] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: : Instructing follower 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:44.839375  3911 raft_consensus.cc:1081] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Signalling peer 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:44.841562  3596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:54090
I20260709 12:22:44.842386  3596 raft_consensus.cc:493] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:44.841531  3495 catalog_manager.cc:5697] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 1 to 2, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 2 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.842772  3596 raft_consensus.cc:3060] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.849112  3596 raft_consensus.cc:515] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.851501  3596 leader_election.cc:290] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.852578  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.852519  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd23c58d49d2463ba8ebe8e135e954e5" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:44.853291  3747 raft_consensus.cc:3055] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:44.853595  3674 raft_consensus.cc:3060] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.853603  3747 raft_consensus.cc:740] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:44.854642  3747 consensus_queue.cc:260] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.855809  3747 raft_consensus.cc:3060] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.860422  3674 raft_consensus.cc:2468] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:44.861728  3565 leader_election.cc:304] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:44.862615  3953 raft_consensus.cc:2804] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:44.862841  3747 raft_consensus.cc:2468] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:44.863201  3953 raft_consensus.cc:697] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.864429  3953 consensus_queue.cc:237] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.873507  3495 catalog_manager.cc:5697] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 1 to 2, leader changed from 915f522cefa74142aa1c5914a877dbbf (127.23.217.67) to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.881891  3953 consensus_queue.cc:1048] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:44.893777  3945 consensus_queue.cc:1048] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:44.903429  3912 raft_consensus.cc:993] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:44.903973  3945 raft_consensus.cc:1081] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Signalling peer 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:44.906021  3673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:22:44.906629  3673 raft_consensus.cc:493] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:44.906955  3673 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.912226  3673 raft_consensus.cc:515] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.914276  3673 leader_election.cc:290] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.915546  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.915761  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:44.916194  3747 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.916568  3596 raft_consensus.cc:3055] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:44.917004  3596 raft_consensus.cc:740] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.918257  3596 consensus_queue.cc:260] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.920579  3596 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.921553  3747 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:44.922722  3638 leader_election.cc:304] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.925017  3853 raft_consensus.cc:2804] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:44.925522  3853 raft_consensus.cc:697] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:44.926381  3853 consensus_queue.cc:237] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.930312  3596 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:44.933374  3496 catalog_manager.cc:5697] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 1 to 2, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 2 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:44.961300  3953 raft_consensus.cc:993] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:44.961848  3953 raft_consensus.cc:1081] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Signalling peer 522b9a515e00491e9c1dbc9a27ac099c to start an election
I20260709 12:22:44.963433  3673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:22:44.964001  3673 raft_consensus.cc:493] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:44.964402  3673 raft_consensus.cc:3060] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.969310  3673 raft_consensus.cc:515] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.971032  3673 leader_election.cc:290] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:44.972143  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:44.972246  3747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab5aba566ee42828060e916fd6b3173" candidate_uuid: "522b9a515e00491e9c1dbc9a27ac099c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:44.972988  3596 raft_consensus.cc:3055] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:44.973093  3747 raft_consensus.cc:3060] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.973367  3596 raft_consensus.cc:740] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:44.974309  3596 consensus_queue.cc:260] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.975551  3596 raft_consensus.cc:3060] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:44.978089  3747 raft_consensus.cc:2468] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:44.979241  3638 leader_election.cc:304] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:44.979944  3853 raft_consensus.cc:2804] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:44.980370  3853 raft_consensus.cc:697] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Leader. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:44.981204  3853 consensus_queue.cc:237] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:44.982250  3596 raft_consensus.cc:2468] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522b9a515e00491e9c1dbc9a27ac099c in term 2.
I20260709 12:22:44.989418  3496 catalog_manager.cc:5697] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c reported cstate change: term changed from 1 to 2, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66). New cstate: current_term: 2 leader_uuid: "522b9a515e00491e9c1dbc9a27ac099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.053156  3885 raft_consensus.cc:993] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: : Instructing follower 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:45.053709  3911 raft_consensus.cc:1081] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Signalling peer 602b6ed4efe04787936c1f77080c61e6 to start an election
I20260709 12:22:45.055436  3596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
 from {username='slave'} at 127.0.0.1:54090
I20260709 12:22:45.056123  3596 raft_consensus.cc:493] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:45.056540  3596 raft_consensus.cc:3060] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:45.064369  3596 raft_consensus.cc:515] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:45.066236  3596 leader_election.cc:290] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Requested vote from peers 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139), 915f522cefa74142aa1c5914a877dbbf (127.23.217.67:34829)
I20260709 12:22:45.067538  3746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
I20260709 12:22:45.068099  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7061fb94422f4b968a3e9364c6b7849a" candidate_uuid: "602b6ed4efe04787936c1f77080c61e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:45.068260  3746 raft_consensus.cc:3055] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:45.068614  3746 raft_consensus.cc:740] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:45.068670  3673 raft_consensus.cc:3060] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:45.069629  3746 consensus_queue.cc:260] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:45.071025  3746 raft_consensus.cc:3060] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:45.074154  3901 consensus_queue.cc:1048] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:45.076503  3673 raft_consensus.cc:2468] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:45.077746  3565 leader_election.cc:304] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 602b6ed4efe04787936c1f77080c61e6; no voters: 
I20260709 12:22:45.078811  3953 raft_consensus.cc:2804] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:45.079404  3953 raft_consensus.cc:697] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Becoming Leader. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:45.080355  3953 consensus_queue.cc:237] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:45.081923  3746 raft_consensus.cc:2468] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602b6ed4efe04787936c1f77080c61e6 in term 2.
I20260709 12:22:45.082036  3853 raft_consensus.cc:993] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: : Instructing follower 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:45.082813  3925 raft_consensus.cc:1081] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Signalling peer 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:45.084650  3746 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:53756
I20260709 12:22:45.085237  3746 raft_consensus.cc:493] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:45.085651  3746 raft_consensus.cc:3060] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:45.095155  3853 consensus_queue.cc:1048] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:45.092089  3495 catalog_manager.cc:5697] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 reported cstate change: term changed from 1 to 2, leader changed from 915f522cefa74142aa1c5914a877dbbf (127.23.217.67) to 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65). New cstate: current_term: 2 leader_uuid: "602b6ed4efe04787936c1f77080c61e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.099164  3746 raft_consensus.cc:515] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:45.100266  3747 raft_consensus.cc:1240] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Rejecting Update request from peer 522b9a515e00491e9c1dbc9a27ac099c for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:45.102901  3746 leader_election.cc:290] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:45.104178  3673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:45.104867  3673 raft_consensus.cc:3055] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:45.105222  3673 raft_consensus.cc:740] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:45.106091  3673 consensus_queue.cc:260] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:45.107115  3673 raft_consensus.cc:3060] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:45.110726  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "935880ce84e04dd1aa02e28cebb1df50" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
W20260709 12:22:45.110560  3853 consensus_queue.cc:1175] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "915f522cefa74142aa1c5914a877dbbf" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 522b9a515e00491e9c1dbc9a27ac099c for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:45.121073  3673 raft_consensus.cc:2468] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 2.
I20260709 12:22:45.122872  3714 leader_election.cc:304] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 602b6ed4efe04787936c1f77080c61e6
I20260709 12:22:45.124248  3911 raft_consensus.cc:2804] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:45.124792  3911 raft_consensus.cc:697] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 2 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:45.125722  3911 consensus_queue.cc:237] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:45.133424  3495 catalog_manager.cc:5697] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 1 to 2, leader changed from 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66) to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 2 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:45.157413  3674 raft_consensus.cc:1275] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.158941  3912 consensus_queue.cc:1048] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.167474  3747 raft_consensus.cc:1275] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.169109  3953 consensus_queue.cc:1048] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.237782  3599 raft_consensus.cc:1275] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.239351  3970 consensus_queue.cc:1048] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.247556  3747 raft_consensus.cc:3060] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:45.254381  3747 raft_consensus.cc:1275] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.256024  3971 consensus_queue.cc:1048] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.257472  3746 raft_consensus.cc:1275] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.259104  3912 consensus_queue.cc:1048] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.267176  3674 raft_consensus.cc:1275] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.268779  3953 consensus_queue.cc:1048] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.403762  3599 raft_consensus.cc:1275] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.405227  3971 consensus_queue.cc:1048] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.413770  3746 raft_consensus.cc:1275] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.415378  3971 consensus_queue.cc:1048] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:45.443305  3599 raft_consensus.cc:1275] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.444723  3971 consensus_queue.cc:1048] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.454031  3746 raft_consensus.cc:1275] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 522b9a515e00491e9c1dbc9a27ac099c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.458509  3971 consensus_queue.cc:1048] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.519397  3746 raft_consensus.cc:1275] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.520771  3953 consensus_queue.cc:1048] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.528435  3674 raft_consensus.cc:1275] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 602b6ed4efe04787936c1f77080c61e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.529867  3953 consensus_queue.cc:1048] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.637284  3674 raft_consensus.cc:1275] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.638681  3911 consensus_queue.cc:1048] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:45.649129  3599 raft_consensus.cc:3060] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:45.662317  3599 raft_consensus.cc:1275] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:45.663760  3975 consensus_queue.cc:1048] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:47.654986  3494 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41338:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:22:47.680405  3494 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41338:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:22:47.741150 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:22:47.884826  3563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:34829, user_credentials={real_user=slave}} blocked reactor thread for 109628us
W20260709 12:22:47.826449  3639 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:43629, user_credentials={real_user=slave}} blocked reactor thread for 51674.4us
W20260709 12:22:47.918807  4000 log.cc:927] Time spent T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:22:47.932382  3999 log.cc:927] Time spent T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: Append to log took a long time: real 0.117s	user 0.001s	sys 0.003s
W20260709 12:22:48.018843  3639 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:43629, user_credentials={real_user=slave}} blocked reactor thread for 65516.4us
I20260709 12:22:48.021093  4034 tablet.cc:1724] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.036329  4038 tablet.cc:1724] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.016785  4024 tablet.cc:1724] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.098197  4048 tablet.cc:1724] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.098555  4039 tablet.cc:1724] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:48.107755  4021 log.cc:927] Time spent T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: Append to log took a long time: real 0.142s	user 0.002s	sys 0.000s
I20260709 12:22:48.107553  4034 tablet.cc:1724] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:48.111522  3638 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:34829, user_credentials={real_user=slave}} blocked reactor thread for 63725.7us
W20260709 12:22:48.104662  3639 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:43629, user_credentials={real_user=slave}} blocked reactor thread for 84949.9us
W20260709 12:22:48.128630  3565 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.66:40139, user_credentials={real_user=slave}} blocked reactor thread for 73728.5us
I20260709 12:22:48.158241  4043 tablet.cc:1724] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.163992  4046 tablet.cc:1724] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:48.158412  4031 log.cc:927] Time spent T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
W20260709 12:22:48.170501  3625 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:7061fb94422f4b968a3e9364c6b7849a)
W20260709 12:22:48.181183  3625 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:7061fb94422f4b968a3e9364c6b7849a)
W20260709 12:22:48.185623  3625 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:7061fb94422f4b968a3e9364c6b7849a)
I20260709 12:22:48.130149  4049 tablet.cc:1724] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.217736  4051 tablet.cc:1724] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.218662  4050 tablet.cc:1724] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.228492  4038 tablet.cc:1724] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.232585  4039 tablet.cc:1724] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.236553  3596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:33300
I20260709 12:22:48.237363  3596 raft_consensus.cc:606] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Received request to transfer leadership to TS 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:48.252097  4034 tablet.cc:1724] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.255280  4047 tablet.cc:1724] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.262346  4053 tablet.cc:1724] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.276283  4024 tablet.cc:1724] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.277763  4049 tablet.cc:1724] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.286953  4048 tablet.cc:1724] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.299695  4049 tablet.cc:1724] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.301537  4050 tablet.cc:1724] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.304493  4039 tablet.cc:1724] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.335018  4053 tablet.cc:1724] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.334192  4051 tablet.cc:1724] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.345129  3672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
mode: GRACEFUL
new_leader_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:52358
I20260709 12:22:48.345978  3672 raft_consensus.cc:606] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Received request to transfer leadership to TS 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:48.348279 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:22:48.347637  4053 tablet.cc:1724] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.349025 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:48.361549  4046 tablet.cc:1724] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.368433  4038 tablet.cc:1724] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:48.369792  3672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756"
dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
mode: GRACEFUL
new_leader_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:52358
I20260709 12:22:48.370580  3672 raft_consensus.cc:606] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Received request to transfer leadership to TS 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:48.375509  3598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712"
dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
mode: GRACEFUL
new_leader_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:33300
I20260709 12:22:48.376091  3598 raft_consensus.cc:606] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Received request to transfer leadership to TS 915f522cefa74142aa1c5914a877dbbf
I20260709 12:22:48.377785 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:48.378962 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:48.390874  3495 catalog_manager.cc:6242] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee: soft_deleted_table [id=3dc6cd60ce1145c59858820cb867e6cc] alter complete (version 1)
I20260709 12:22:48.652009  3953 raft_consensus.cc:993] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.652783  4025 raft_consensus.cc:1081] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Signalling peer 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.654824  3744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:53754
I20260709 12:22:48.655408  3744 raft_consensus.cc:493] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:48.655689  3744 raft_consensus.cc:3060] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:48.661840  3744 raft_consensus.cc:515] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.664137  3744 leader_election.cc:290] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 2 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:48.664906  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:48.665462  3672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b5a56d70a5451f94c458da45debd72" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:48.665709  3598 raft_consensus.cc:3055] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:48.666061  3598 raft_consensus.cc:740] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:48.666117  3672 raft_consensus.cc:3060] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:48.667003  3598 consensus_queue.cc:260] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.668377  3598 raft_consensus.cc:3060] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:48.674582  3672 raft_consensus.cc:2468] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 2.
I20260709 12:22:48.675978  3714 leader_election.cc:304] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522b9a515e00491e9c1dbc9a27ac099c, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:48.676720  3598 raft_consensus.cc:2468] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 2.
I20260709 12:22:48.676950  4002 raft_consensus.cc:2804] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:48.678346  4002 raft_consensus.cc:697] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 2 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:48.679430  4002 consensus_queue.cc:237] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.689932  3497 catalog_manager.cc:5697] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 1 to 2, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 2 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:48.753619  4045 raft_consensus.cc:993] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: : Instructing follower 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.754288  4035 raft_consensus.cc:1081] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Signalling peer 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.756052  3744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:53756
I20260709 12:22:48.756605  3744 raft_consensus.cc:493] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:48.756903  3744 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.763037  3953 raft_consensus.cc:993] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6: : Instructing follower 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.763684  4025 raft_consensus.cc:1081] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Signalling peer 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.764230  3744 raft_consensus.cc:515] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.765465  3743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:53754
I20260709 12:22:48.766186  3743 raft_consensus.cc:493] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:48.766590  3743 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.767256  3598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:48.767776  3598 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.766549  3744 leader_election.cc:290] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 3 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:48.771973  3672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f61e3b9e450427f856143ce283b4eda" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:48.772706  3672 raft_consensus.cc:3055] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:48.773044  3672 raft_consensus.cc:740] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:48.773707  3743 raft_consensus.cc:515] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.773883  3672 consensus_queue.cc:260] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.774804  3598 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 3.
I20260709 12:22:48.775452  3672 raft_consensus.cc:3060] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.776139  3713 leader_election.cc:304] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:48.775887  3743 leader_election.cc:290] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 3 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:48.777066  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:48.777304  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8df4615f2eae4a22b1dce75d1a544712" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:48.777732  4002 raft_consensus.cc:2804] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:48.777966  3596 raft_consensus.cc:3055] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:48.778324  3596 raft_consensus.cc:740] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 602b6ed4efe04787936c1f77080c61e6, State: Running, Role: LEADER
I20260709 12:22:48.778380  4002 raft_consensus.cc:697] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 3 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:48.779191  3596 consensus_queue.cc:260] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.777705  3674 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.779289  4002 consensus_queue.cc:237] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.780539  3596 raft_consensus.cc:3060] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.784251  3672 raft_consensus.cc:2468] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 3.
I20260709 12:22:48.787490  3596 raft_consensus.cc:2468] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 3.
I20260709 12:22:48.788605  3713 leader_election.cc:304] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:48.788734  3674 raft_consensus.cc:2468] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 3.
I20260709 12:22:48.789510  4002 raft_consensus.cc:2804] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:48.789127  3497 catalog_manager.cc:5697] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 2 to 3, leader changed from 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66) to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 3 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:48.790182  4002 raft_consensus.cc:697] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 3 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:48.791193  4002 consensus_queue.cc:237] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.802764  3497 catalog_manager.cc:5697] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 2 to 3, leader changed from 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65) to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 3 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:48.839540  4045 raft_consensus.cc:993] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c: : Instructing follower 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.840142  4035 raft_consensus.cc:1081] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Signalling peer 915f522cefa74142aa1c5914a877dbbf to start an election
I20260709 12:22:48.841750  3743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756"
dest_uuid: "915f522cefa74142aa1c5914a877dbbf"
 from {username='slave'} at 127.0.0.1:53756
I20260709 12:22:48.842316  3743 raft_consensus.cc:493] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:48.842694  3743 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.847683  3743 raft_consensus.cc:515] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.849404  3743 leader_election.cc:290] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 3 election: Requested vote from peers 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629), 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139)
I20260709 12:22:48.850319  3596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "602b6ed4efe04787936c1f77080c61e6"
I20260709 12:22:48.850571  3674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d9ba63cc334a0d960ebcf224c79756" candidate_uuid: "915f522cefa74142aa1c5914a877dbbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "522b9a515e00491e9c1dbc9a27ac099c"
I20260709 12:22:48.850972  3596 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.851078  3674 raft_consensus.cc:3055] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:48.851440  3674 raft_consensus.cc:740] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522b9a515e00491e9c1dbc9a27ac099c, State: Running, Role: LEADER
I20260709 12:22:48.852406  3674 consensus_queue.cc:260] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.853657  3674 raft_consensus.cc:3060] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.857607  3596 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 3.
I20260709 12:22:48.858853  3713 leader_election.cc:304] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602b6ed4efe04787936c1f77080c61e6, 915f522cefa74142aa1c5914a877dbbf; no voters: 
I20260709 12:22:48.859766  4002 raft_consensus.cc:2804] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:48.860227  3674 raft_consensus.cc:2468] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915f522cefa74142aa1c5914a877dbbf in term 3.
I20260709 12:22:48.860311  4002 raft_consensus.cc:697] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 3 LEADER]: Becoming Leader. State: Replica: 915f522cefa74142aa1c5914a877dbbf, State: Running, Role: LEADER
I20260709 12:22:48.862054  4002 consensus_queue.cc:237] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } }
I20260709 12:22:48.871908  3497 catalog_manager.cc:5697] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf reported cstate change: term changed from 2 to 3, leader changed from 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66) to 915f522cefa74142aa1c5914a877dbbf (127.23.217.67). New cstate: current_term: 3 leader_uuid: "915f522cefa74142aa1c5914a877dbbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "915f522cefa74142aa1c5914a877dbbf" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 34829 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:49.080667  3596 raft_consensus.cc:1275] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:49.082023  4002 consensus_queue.cc:1048] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.090952  3674 raft_consensus.cc:1275] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:22:49.092866  4061 consensus_queue.cc:1048] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.210296  3674 raft_consensus.cc:1275] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.211912  4061 consensus_queue.cc:1048] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.224490  3596 raft_consensus.cc:1275] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.226183  4061 consensus_queue.cc:1048] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.307435  3596 raft_consensus.cc:1275] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:49.308723  4061 consensus_queue.cc:1048] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:49.319314  3672 raft_consensus.cc:1275] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:22:49.321175  4059 consensus_queue.cc:1048] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:49.363353  3674 raft_consensus.cc:1275] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.364794  4059 consensus_queue.cc:1048] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "522b9a515e00491e9c1dbc9a27ac099c" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.374701  3596 raft_consensus.cc:1275] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Refusing update from remote peer 915f522cefa74142aa1c5914a877dbbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.377449  4068 consensus_queue.cc:1048] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "602b6ed4efe04787936c1f77080c61e6" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.380647 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:49.390285 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:49.390863 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:49.399981 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:49.400511 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:50.401646 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:50.409144 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:50.409842 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:50.418238 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:50.418896 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:51.420074 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:51.426774 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:51.427183 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:51.432937 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:51.433478 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:52.434929 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:52.441022 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:52.441568 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:52.446739 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:52.447209 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:53.448372 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:53.454238 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:53.454782 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:53.459875 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:53.460305 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:54.461751 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:54.473026 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:54.473619 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:54.480348 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:54.480842 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:55.482134 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:55.487545 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:55.488049 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:55.492930 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:55.493342 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:56.494403 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:56.499645 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:56.500173 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:56.505800 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:56.506337 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:57.507512 24421 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:22:57.514267 24421 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:22:57.514837 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:57.519621 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:57.520038 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:58.590174 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
I20260709 12:22:58.639266 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:58.640076 24421 tablet_replica.cc:333] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.641130 24421 raft_consensus.cc:2243] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.641849 24421 raft_consensus.cc:2272] T 5198ba68cd8342098f9b5e712b0bf1fc P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.644740 24421 tablet_replica.cc:333] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.645411 24421 raft_consensus.cc:2243] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.645961 24421 raft_consensus.cc:2272] T 25a07ac21d3946dca3d7416afaee7243 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.648736 24421 tablet_replica.cc:333] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.649331 24421 raft_consensus.cc:2243] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.650341 24421 raft_consensus.cc:2272] T 6052b004c40f4059a30d727097987503 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.652992 24421 tablet_replica.cc:333] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.653575 24421 raft_consensus.cc:2243] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.654371 24421 raft_consensus.cc:2272] T dd23c58d49d2463ba8ebe8e135e954e5 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.657495 24421 tablet_replica.cc:333] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.658444 24421 raft_consensus.cc:2243] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:22:58.658612  3713 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
I20260709 12:22:58.659873 24421 raft_consensus.cc:2272] T 7061fb94422f4b968a3e9364c6b7849a P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:58.663276  3713 consensus_peers.cc:597] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.664016 24421 tablet_replica.cc:333] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.664762 24421 raft_consensus.cc:2243] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.665280 24421 raft_consensus.cc:2272] T 38b5a56d70a5451f94c458da45debd72 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.667797 24421 tablet_replica.cc:333] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.668380 24421 raft_consensus.cc:2243] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.668857 24421 raft_consensus.cc:2272] T d1ccf46e805348be875b04ab724ec30e P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.671456 24421 tablet_replica.cc:333] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.672081 24421 raft_consensus.cc:2243] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.672665 24421 raft_consensus.cc:2272] T 6ab5aba566ee42828060e916fd6b3173 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.675103 24421 tablet_replica.cc:333] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.675879 24421 raft_consensus.cc:2243] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.676599 24421 raft_consensus.cc:2272] T 8df4615f2eae4a22b1dce75d1a544712 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.681742 24421 tablet_replica.cc:333] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.682932 24421 raft_consensus.cc:2243] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.684067 24421 raft_consensus.cc:2272] T 4f61e3b9e450427f856143ce283b4eda P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:58.684731  3713 consensus_peers.cc:597] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.688979 24421 tablet_replica.cc:333] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
W20260709 12:22:58.689356  3639 consensus_peers.cc:597] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.689976 24421 raft_consensus.cc:2243] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.691094 24421 raft_consensus.cc:2272] T 1d4852bb48c749abb6c9c2bc6c786d02 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.694060 24421 tablet_replica.cc:333] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.694792 24421 raft_consensus.cc:2243] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.695413 24421 raft_consensus.cc:2272] T 086558ab35914c12869f94ad34540292 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.698746 24421 tablet_replica.cc:333] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.699699 24421 raft_consensus.cc:2243] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.700397 24421 raft_consensus.cc:2272] T f78d8d43e1af454e9be9691159fa7525 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.703392 24421 tablet_replica.cc:333] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.704057 24421 raft_consensus.cc:2243] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.705144 24421 raft_consensus.cc:2272] T 4da3e8ce02cc4f2589ce772d40d87fbc P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.707998 24421 tablet_replica.cc:333] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.708652 24421 raft_consensus.cc:2243] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.709313 24421 raft_consensus.cc:2272] T 935880ce84e04dd1aa02e28cebb1df50 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.712236 24421 tablet_replica.cc:333] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.712898 24421 raft_consensus.cc:2243] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.713558 24421 raft_consensus.cc:2272] T 35d9ba63cc334a0d960ebcf224c79756 P 602b6ed4efe04787936c1f77080c61e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.716290 24421 tablet_replica.cc:333] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.717067 24421 raft_consensus.cc:2243] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.718322 24421 raft_consensus.cc:2272] T ce0ea8bd86f245ad8264344eb0fe2c92 P 602b6ed4efe04787936c1f77080c61e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.721525 24421 tablet_replica.cc:333] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6: stopping tablet replica
I20260709 12:22:58.722164 24421 raft_consensus.cc:2243] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.723278 24421 raft_consensus.cc:2272] T 6827650806d94c81b65b6bf932270220 P 602b6ed4efe04787936c1f77080c61e6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:58.754439  3639 consensus_peers.cc:597] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:58.783052  3639 consensus_peers.cc:597] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.784009 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
W20260709 12:22:58.834782  3713 consensus_peers.cc:597] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.838335 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
W20260709 12:22:58.859673  3639 consensus_peers.cc:597] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.882714 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:58.883845 24421 tablet_replica.cc:333] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.885099 24421 raft_consensus.cc:2243] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.886480 24421 raft_consensus.cc:2272] T 5198ba68cd8342098f9b5e712b0bf1fc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:58.888797  3639 consensus_peers.cc:597] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.889897 24421 tablet_replica.cc:333] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.890738 24421 raft_consensus.cc:2243] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.891587 24421 raft_consensus.cc:2272] T 25a07ac21d3946dca3d7416afaee7243 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.893855 24421 tablet_replica.cc:333] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.894434 24421 raft_consensus.cc:2243] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.894937 24421 raft_consensus.cc:2272] T 38b5a56d70a5451f94c458da45debd72 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.897262 24421 tablet_replica.cc:333] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.897948 24421 raft_consensus.cc:2243] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.898742 24421 raft_consensus.cc:2272] T dd23c58d49d2463ba8ebe8e135e954e5 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:58.901732  3714 consensus_peers.cc:597] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf -> Peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Couldn't send request to peer 522b9a515e00491e9c1dbc9a27ac099c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.901916 24421 tablet_replica.cc:333] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.902714 24421 raft_consensus.cc:2243] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.903213 24421 raft_consensus.cc:2272] T 7061fb94422f4b968a3e9364c6b7849a P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.905380 24421 tablet_replica.cc:333] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.905913 24421 raft_consensus.cc:2243] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.906813 24421 raft_consensus.cc:2272] T 6ab5aba566ee42828060e916fd6b3173 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.909009 24421 tablet_replica.cc:333] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.909545 24421 raft_consensus.cc:2243] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.910120 24421 raft_consensus.cc:2272] T 8df4615f2eae4a22b1dce75d1a544712 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.913273 24421 tablet_replica.cc:333] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.913880 24421 raft_consensus.cc:2243] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:22:58.914090  3713 consensus_peers.cc:597] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.914403 24421 raft_consensus.cc:2272] T 6827650806d94c81b65b6bf932270220 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.916893 24421 tablet_replica.cc:333] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.917469 24421 raft_consensus.cc:2243] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.917917 24421 raft_consensus.cc:2272] T ce0ea8bd86f245ad8264344eb0fe2c92 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.920262 24421 tablet_replica.cc:333] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.920800 24421 raft_consensus.cc:2243] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.921337 24421 raft_consensus.cc:2272] T 935880ce84e04dd1aa02e28cebb1df50 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.923642 24421 tablet_replica.cc:333] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.924239 24421 raft_consensus.cc:2243] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.924733 24421 raft_consensus.cc:2272] T 35d9ba63cc334a0d960ebcf224c79756 P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.927031 24421 tablet_replica.cc:333] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.927544 24421 raft_consensus.cc:2243] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.928089 24421 raft_consensus.cc:2272] T 6052b004c40f4059a30d727097987503 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.930204 24421 tablet_replica.cc:333] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.930807 24421 raft_consensus.cc:2243] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.931279 24421 raft_consensus.cc:2272] T 4f61e3b9e450427f856143ce283b4eda P 522b9a515e00491e9c1dbc9a27ac099c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.933460 24421 tablet_replica.cc:333] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.933987 24421 raft_consensus.cc:2243] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.934895 24421 raft_consensus.cc:2272] T 1d4852bb48c749abb6c9c2bc6c786d02 P 522b9a515e00491e9c1dbc9a27ac099c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.937392 24421 tablet_replica.cc:333] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.938112 24421 raft_consensus.cc:2243] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.939088 24421 raft_consensus.cc:2272] T 086558ab35914c12869f94ad34540292 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.941833 24421 tablet_replica.cc:333] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.942396 24421 raft_consensus.cc:2243] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:58.943284 24421 raft_consensus.cc:2272] T d1ccf46e805348be875b04ab724ec30e P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.945423 24421 tablet_replica.cc:333] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.945932 24421 raft_consensus.cc:2243] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.946434 24421 raft_consensus.cc:2272] T f78d8d43e1af454e9be9691159fa7525 P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:58.948475 24421 tablet_replica.cc:333] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c: stopping tablet replica
I20260709 12:22:58.948958 24421 raft_consensus.cc:2243] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:58.949450 24421 raft_consensus.cc:2272] T 4da3e8ce02cc4f2589ce772d40d87fbc P 522b9a515e00491e9c1dbc9a27ac099c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:58.958705  3714 consensus_peers.cc:597] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf -> Peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Couldn't send request to peer 522b9a515e00491e9c1dbc9a27ac099c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:58.997386 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
W20260709 12:22:59.003537  3713 consensus_peers.cc:597] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:59.007498  3714 consensus_peers.cc:597] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf -> Peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Couldn't send request to peer 522b9a515e00491e9c1dbc9a27ac099c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:40139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:59.009666  3714 consensus_peers.cc:597] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf -> Peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Couldn't send request to peer 522b9a515e00491e9c1dbc9a27ac099c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:40139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:59.036973  3713 consensus_peers.cc:597] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf -> Peer 602b6ed4efe04787936c1f77080c61e6 (127.23.217.65:43629): Couldn't send request to peer 602b6ed4efe04787936c1f77080c61e6. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:59.048678 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
I20260709 12:22:59.085916 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:59.087041 24421 tablet_replica.cc:333] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.088050 24421 raft_consensus.cc:2243] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.089020 24421 raft_consensus.cc:2272] T 25a07ac21d3946dca3d7416afaee7243 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:59.092367  3714 consensus_peers.cc:597] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf -> Peer 522b9a515e00491e9c1dbc9a27ac099c (127.23.217.66:40139): Couldn't send request to peer 522b9a515e00491e9c1dbc9a27ac099c. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:40139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:59.092624 24421 tablet_replica.cc:333] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.093430 24421 raft_consensus.cc:2243] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.093958 24421 raft_consensus.cc:2272] T 5198ba68cd8342098f9b5e712b0bf1fc P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.096323 24421 tablet_replica.cc:333] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.096987 24421 raft_consensus.cc:2243] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.097478 24421 raft_consensus.cc:2272] T 6052b004c40f4059a30d727097987503 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.099668 24421 tablet_replica.cc:333] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.100231 24421 raft_consensus.cc:2243] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:59.101084 24421 raft_consensus.cc:2272] T 38b5a56d70a5451f94c458da45debd72 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.103492 24421 tablet_replica.cc:333] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.104074 24421 raft_consensus.cc:2243] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.104583 24421 raft_consensus.cc:2272] T dd23c58d49d2463ba8ebe8e135e954e5 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.107003 24421 tablet_replica.cc:333] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.107595 24421 raft_consensus.cc:2243] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.108083 24421 raft_consensus.cc:2272] T 7061fb94422f4b968a3e9364c6b7849a P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.110112 24421 tablet_replica.cc:333] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.110743 24421 raft_consensus.cc:2243] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.111181 24421 raft_consensus.cc:2272] T d1ccf46e805348be875b04ab724ec30e P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.113471 24421 tablet_replica.cc:333] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.113976 24421 raft_consensus.cc:2243] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:59.114939 24421 raft_consensus.cc:2272] T f78d8d43e1af454e9be9691159fa7525 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.117141 24421 tablet_replica.cc:333] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.117635 24421 raft_consensus.cc:2243] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.118086 24421 raft_consensus.cc:2272] T 4da3e8ce02cc4f2589ce772d40d87fbc P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.120254 24421 tablet_replica.cc:333] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.120783 24421 raft_consensus.cc:2243] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.121229 24421 raft_consensus.cc:2272] T 6827650806d94c81b65b6bf932270220 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.123533 24421 tablet_replica.cc:333] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.124023 24421 raft_consensus.cc:2243] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:59.124912 24421 raft_consensus.cc:2272] T 4f61e3b9e450427f856143ce283b4eda P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.127089 24421 tablet_replica.cc:333] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.127571 24421 raft_consensus.cc:2243] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.127995 24421 raft_consensus.cc:2272] T 1d4852bb48c749abb6c9c2bc6c786d02 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.130095 24421 tablet_replica.cc:333] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.130642 24421 raft_consensus.cc:2243] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.131091 24421 raft_consensus.cc:2272] T 086558ab35914c12869f94ad34540292 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.133173 24421 tablet_replica.cc:333] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.133634 24421 raft_consensus.cc:2243] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.134054 24421 raft_consensus.cc:2272] T 6ab5aba566ee42828060e916fd6b3173 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.136071 24421 tablet_replica.cc:333] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.136543 24421 raft_consensus.cc:2243] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:59.137379 24421 raft_consensus.cc:2272] T 8df4615f2eae4a22b1dce75d1a544712 P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.139622 24421 tablet_replica.cc:333] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.140141 24421 raft_consensus.cc:2243] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:59.140548 24421 raft_consensus.cc:2272] T ce0ea8bd86f245ad8264344eb0fe2c92 P 915f522cefa74142aa1c5914a877dbbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.142820 24421 tablet_replica.cc:333] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.143368 24421 raft_consensus.cc:2243] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:59.144274 24421 raft_consensus.cc:2272] T 935880ce84e04dd1aa02e28cebb1df50 P 915f522cefa74142aa1c5914a877dbbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.146559 24421 tablet_replica.cc:333] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf: stopping tablet replica
I20260709 12:22:59.147074 24421 raft_consensus.cc:2243] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:59.148027 24421 raft_consensus.cc:2272] T 35d9ba63cc334a0d960ebcf224c79756 P 915f522cefa74142aa1c5914a877dbbf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.192894 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
I20260709 12:22:59.236048 24421 master.cc:562] Master@127.23.217.126:41283 shutting down...
I20260709 12:22:59.267530 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:59.268373 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:59.268784 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P c27bc8b6b87b4bd492fe58b423701fee: stopping tablet replica
I20260709 12:22:59.292660 24421 master.cc:584] Master@127.23.217.126:41283 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21318 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:22:59.361583 24421 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.23.217.126:34683
I20260709 12:22:59.363379 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:59.369927  4097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:59.369920  4096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:59.370954  4099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:59.372726 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:59.373766 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:59.373979 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:59.374154 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599779374135 us; error 0 us; skew 500 ppm
I20260709 12:22:59.374739 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:59.377326 24421 webserver.cc:533] Webserver started at http://127.23.217.126:36055/ using document root <none> and password file <none>
I20260709 12:22:59.377840 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:59.378063 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:59.378340 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:59.379735 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/master-0-root/instance:
uuid: "7987a990f5aa4865bf369f5ef381659a"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.384577 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:59.388310  4104 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.389231 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:22:59.389565 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/master-0-root
uuid: "7987a990f5aa4865bf369f5ef381659a"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.389855 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:59.403687 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:59.404879 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:59.446161 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.126:34683
I20260709 12:22:59.446369  4155 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.126:34683 every 8 connection(s)
I20260709 12:22:59.450668  4156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:59.464701  4156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a: Bootstrap starting.
I20260709 12:22:59.470418  4156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:59.475683  4156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a: No bootstrap required, opened a new log
I20260709 12:22:59.478353  4156 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7987a990f5aa4865bf369f5ef381659a" member_type: VOTER }
I20260709 12:22:59.478910  4156 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:59.479205  4156 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7987a990f5aa4865bf369f5ef381659a, State: Initialized, Role: FOLLOWER
I20260709 12:22:59.479959  4156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7987a990f5aa4865bf369f5ef381659a" member_type: VOTER }
I20260709 12:22:59.480470  4156 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:59.480708  4156 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:59.481019  4156 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:59.486812  4156 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7987a990f5aa4865bf369f5ef381659a" member_type: VOTER }
I20260709 12:22:59.487411  4156 leader_election.cc:304] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7987a990f5aa4865bf369f5ef381659a; no voters: 
I20260709 12:22:59.488754  4156 leader_election.cc:290] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:59.489374  4159 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:59.490958  4159 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 1 LEADER]: Becoming Leader. State: Replica: 7987a990f5aa4865bf369f5ef381659a, State: Running, Role: LEADER
I20260709 12:22:59.491706  4159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7987a990f5aa4865bf369f5ef381659a" member_type: VOTER }
I20260709 12:22:59.492475  4156 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:59.496950  4161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7987a990f5aa4865bf369f5ef381659a. Latest consensus state: current_term: 1 leader_uuid: "7987a990f5aa4865bf369f5ef381659a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7987a990f5aa4865bf369f5ef381659a" member_type: VOTER } }
I20260709 12:22:59.497737  4161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:59.497572  4160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7987a990f5aa4865bf369f5ef381659a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7987a990f5aa4865bf369f5ef381659a" member_type: VOTER } }
I20260709 12:22:59.499070  4160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:59.503953  4166 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:59.510512  4166 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:59.514103 24421 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:59.521682  4166 catalog_manager.cc:1383] Generated new cluster ID: dd281d6f56ba43e9b9535880e989e809
I20260709 12:22:59.522045  4166 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:59.536490  4166 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:59.537977  4166 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:59.551450  4166 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a: Generated new TSK 0
I20260709 12:22:59.552178  4166 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:59.580890 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:59.587518  4177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:59.588518  4178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:59.590145  4180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:59.591027 24421 server_base.cc:1061] running on GCE node
I20260709 12:22:59.592317 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:59.592531 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:59.592747 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599779592729 us; error 0 us; skew 500 ppm
I20260709 12:22:59.593647 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:59.596537 24421 webserver.cc:533] Webserver started at http://127.23.217.65:35075/ using document root <none> and password file <none>
I20260709 12:22:59.597106 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:59.597347 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:59.597664 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:59.599046 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-0-root/instance:
uuid: "50159f6246ed4d19bc4169a9c142b98d"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.604297 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:22:59.608780  4185 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.610014 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:22:59.610422 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-0-root
uuid: "50159f6246ed4d19bc4169a9c142b98d"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.610899 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:59.645126 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:59.646307 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:59.647929 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:59.650429 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:59.650664 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.651027 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:59.651232 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.700251 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.65:36539
I20260709 12:22:59.700477  4247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.65:36539 every 8 connection(s)
I20260709 12:22:59.705787 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:59.713316  4252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:59.714829  4253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:59.719071 24421 server_base.cc:1061] running on GCE node
W20260709 12:22:59.719836  4255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:59.721165 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:59.721472 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:59.721694 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599779721670 us; error 0 us; skew 500 ppm
I20260709 12:22:59.722573 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:59.725036  4248 heartbeater.cc:344] Connected to a master server at 127.23.217.126:34683
I20260709 12:22:59.725591  4248 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:59.725867 24421 webserver.cc:533] Webserver started at http://127.23.217.66:43273/ using document root <none> and password file <none>
I20260709 12:22:59.726693 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:59.726701  4248 heartbeater.cc:507] Master 127.23.217.126:34683 requested a full tablet report, sending...
I20260709 12:22:59.727236 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:59.727839 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:59.729205 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-1-root/instance:
uuid: "53ebf9844ebb4152846b6e9fd1249a17"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.729984  4121 ts_manager.cc:194] Registered new tserver with Master: 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539)
I20260709 12:22:59.732671  4121 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40224
I20260709 12:22:59.735553 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:22:59.739454  4260 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.740437 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:59.740778 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-1-root
uuid: "53ebf9844ebb4152846b6e9fd1249a17"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.741030 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:59.755427 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:59.756980 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:59.758812 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:59.761430 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:59.761680 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.761955 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:59.762136 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.806818 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.66:42909
I20260709 12:22:59.807037  4322 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.66:42909 every 8 connection(s)
I20260709 12:22:59.811775 24421 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:59.819697  4326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:59.821606  4327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:59.823055 24421 server_base.cc:1061] running on GCE node
W20260709 12:22:59.824671  4329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:59.825827 24421 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:59.826118 24421 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:59.826359 24421 hybrid_clock.cc:648] HybridClock initialized: now 1783599779826340 us; error 0 us; skew 500 ppm
I20260709 12:22:59.826537  4323 heartbeater.cc:344] Connected to a master server at 127.23.217.126:34683
I20260709 12:22:59.827060 24421 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:59.827080  4323 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:59.828297  4323 heartbeater.cc:507] Master 127.23.217.126:34683 requested a full tablet report, sending...
I20260709 12:22:59.829965 24421 webserver.cc:533] Webserver started at http://127.23.217.67:40025/ using document root <none> and password file <none>
I20260709 12:22:59.830683 24421 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:59.830937 24421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:59.830830  4121 ts_manager.cc:194] Registered new tserver with Master: 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:22:59.831396 24421 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:59.832791  4121 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40238
I20260709 12:22:59.832954 24421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-2-root/instance:
uuid: "3e5c80c10c344270b0d22ed21ccb1274"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.838117 24421 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:22:59.843611  4334 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.844689 24421 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:59.845146 24421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-2-root
uuid: "3e5c80c10c344270b0d22ed21ccb1274"
format_stamp: "Formatted at 2026-07-09 12:22:59 on dist-test-slave-lg8f"
I20260709 12:22:59.845463 24421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskuMktri/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599362490993-24421-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:59.864966 24421 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:59.866454 24421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:59.868273 24421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:59.870828 24421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:59.871021 24421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.871348 24421 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:59.871495 24421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:59.919866 24421 rpc_server.cc:307] RPC server started. Bound to: 127.23.217.67:33495
I20260709 12:22:59.919970  4396 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.23.217.67:33495 every 8 connection(s)
I20260709 12:22:59.936281  4397 heartbeater.cc:344] Connected to a master server at 127.23.217.126:34683
I20260709 12:22:59.936865  4397 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:59.937788  4397 heartbeater.cc:507] Master 127.23.217.126:34683 requested a full tablet report, sending...
I20260709 12:22:59.940147  4121 ts_manager.cc:194] Registered new tserver with Master: 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:22:59.941021 24421 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017527319s
I20260709 12:22:59.942025  4121 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40250
I20260709 12:22:59.969525  4121 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40256:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:59.972335  4121 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:00.051563  4362 tablet_service.cc:1511] Processing CreateTablet for tablet 26a23c16600842d39941c8d6416ebf54 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:00.051996  4361 tablet_service.cc:1511] Processing CreateTablet for tablet f936823aef084ec084d006f0af36a144 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:00.053120  4362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a23c16600842d39941c8d6416ebf54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.053745  4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f936823aef084ec084d006f0af36a144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.056691  4360 tablet_service.cc:1511] Processing CreateTablet for tablet a7cac0b3ada94e2ab2953640133556f7 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:00.056808  4288 tablet_service.cc:1511] Processing CreateTablet for tablet 26a23c16600842d39941c8d6416ebf54 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:00.058732  4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7cac0b3ada94e2ab2953640133556f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.061094  4287 tablet_service.cc:1511] Processing CreateTablet for tablet f936823aef084ec084d006f0af36a144 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:00.061771  4359 tablet_service.cc:1511] Processing CreateTablet for tablet 5a3be6e6308145f7a32b2ede29daafa4 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:00.063290  4359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3be6e6308145f7a32b2ede29daafa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.065709  4286 tablet_service.cc:1511] Processing CreateTablet for tablet a7cac0b3ada94e2ab2953640133556f7 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:00.068977  4358 tablet_service.cc:1511] Processing CreateTablet for tablet 51aba7e903494170beedc9ed8d56e026 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:00.070474  4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51aba7e903494170beedc9ed8d56e026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.074761  4284 tablet_service.cc:1511] Processing CreateTablet for tablet 51aba7e903494170beedc9ed8d56e026 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:00.071079  4285 tablet_service.cc:1511] Processing CreateTablet for tablet 5a3be6e6308145f7a32b2ede29daafa4 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:00.072736  4357 tablet_service.cc:1511] Processing CreateTablet for tablet d2ebc79e324843fc9926cf5699e20916 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:00.081502  4283 tablet_service.cc:1511] Processing CreateTablet for tablet d2ebc79e324843fc9926cf5699e20916 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:00.082015  4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ebc79e324843fc9926cf5699e20916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.086171  4282 tablet_service.cc:1511] Processing CreateTablet for tablet 722efbbd5c784722a3277e0fb9453f21 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:00.087185  4281 tablet_service.cc:1511] Processing CreateTablet for tablet 291c358aca9c4f399b45bfc8f1884c32 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:00.094128  4213 tablet_service.cc:1511] Processing CreateTablet for tablet 26a23c16600842d39941c8d6416ebf54 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:00.087054  4356 tablet_service.cc:1511] Processing CreateTablet for tablet 722efbbd5c784722a3277e0fb9453f21 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:00.095806  4213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a23c16600842d39941c8d6416ebf54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.089784  4355 tablet_service.cc:1511] Processing CreateTablet for tablet 291c358aca9c4f399b45bfc8f1884c32 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:00.091331  4280 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:00.097081  4212 tablet_service.cc:1511] Processing CreateTablet for tablet f936823aef084ec084d006f0af36a144 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:00.092329  4354 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:00.098428  4212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f936823aef084ec084d006f0af36a144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.058777  4288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a23c16600842d39941c8d6416ebf54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.101035  4287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f936823aef084ec084d006f0af36a144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.104029  4211 tablet_service.cc:1511] Processing CreateTablet for tablet a7cac0b3ada94e2ab2953640133556f7 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:00.105474  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7cac0b3ada94e2ab2953640133556f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.107149  4284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51aba7e903494170beedc9ed8d56e026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.111382  4281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 291c358aca9c4f399b45bfc8f1884c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.115757  4210 tablet_service.cc:1511] Processing CreateTablet for tablet 5a3be6e6308145f7a32b2ede29daafa4 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:00.117102  4282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722efbbd5c784722a3277e0fb9453f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.117430  4210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3be6e6308145f7a32b2ede29daafa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.118062  4286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7cac0b3ada94e2ab2953640133556f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.123068  4280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7e9dd20dd6474e9fdfbd4a4b4fbb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.127007  4285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3be6e6308145f7a32b2ede29daafa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.131098  4283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ebc79e324843fc9926cf5699e20916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.132601  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 51aba7e903494170beedc9ed8d56e026 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:00.134351  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51aba7e903494170beedc9ed8d56e026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.135277  4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7e9dd20dd6474e9fdfbd4a4b4fbb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.137419  4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722efbbd5c784722a3277e0fb9453f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.139408  4208 tablet_service.cc:1511] Processing CreateTablet for tablet d2ebc79e324843fc9926cf5699e20916 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:00.140693  4355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 291c358aca9c4f399b45bfc8f1884c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.141049  4208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ebc79e324843fc9926cf5699e20916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.178401  4207 tablet_service.cc:1511] Processing CreateTablet for tablet 722efbbd5c784722a3277e0fb9453f21 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:00.180143  4207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722efbbd5c784722a3277e0fb9453f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.188373  4206 tablet_service.cc:1511] Processing CreateTablet for tablet 291c358aca9c4f399b45bfc8f1884c32 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:00.189883  4206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 291c358aca9c4f399b45bfc8f1884c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.209812  4205 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 (DEFAULT_TABLE table=test-workload [id=973ca3d08f014b84bacf6e604ef86659]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:00.211426  4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7e9dd20dd6474e9fdfbd4a4b4fbb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:00.227285  4414 tablet_bootstrap.cc:492] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.280967  4412 tablet_bootstrap.cc:492] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:00.289248  4412 tablet_bootstrap.cc:654] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.298105  4414 tablet_bootstrap.cc:654] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.312304  4413 tablet_bootstrap.cc:492] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.324977  4412 tablet_bootstrap.cc:492] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:00.325119  4414 tablet_bootstrap.cc:492] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.325861  4412 ts_tablet_manager.cc:1403] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.009s
I20260709 12:23:00.325883  4414 ts_tablet_manager.cc:1403] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.000s
I20260709 12:23:00.326910  4413 tablet_bootstrap.cc:654] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.328851  4414 raft_consensus.cc:359] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.329491  4414 raft_consensus.cc:385] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.329833  4414 raft_consensus.cc:740] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.329558  4412 raft_consensus.cc:359] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.330693  4414 consensus_queue.cc:260] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.335445  4412 raft_consensus.cc:385] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.335978  4412 raft_consensus.cc:740] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.337061  4412 consensus_queue.cc:260] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.339221  4413 tablet_bootstrap.cc:492] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.339999  4413 ts_tablet_manager.cc:1403] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.011s
I20260709 12:23:00.344154  4413 raft_consensus.cc:359] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.345144  4413 raft_consensus.cc:385] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.345530  4413 raft_consensus.cc:740] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.347030  4413 consensus_queue.cc:260] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.347729  4323 heartbeater.cc:499] Master 127.23.217.126:34683 was elected leader, sending a full tablet report...
I20260709 12:23:00.348349  4414 ts_tablet_manager.cc:1434] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.013s
I20260709 12:23:00.349656  4414 tablet_bootstrap.cc:492] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.351997  4397 heartbeater.cc:499] Master 127.23.217.126:34683 was elected leader, sending a full tablet report...
I20260709 12:23:00.352957  4412 ts_tablet_manager.cc:1434] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.005s
I20260709 12:23:00.354252  4412 tablet_bootstrap.cc:492] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:00.356642  4414 tablet_bootstrap.cc:654] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.361076  4412 tablet_bootstrap.cc:654] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.366704  4248 heartbeater.cc:499] Master 127.23.217.126:34683 was elected leader, sending a full tablet report...
I20260709 12:23:00.367120  4413 ts_tablet_manager.cc:1434] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.027s	user 0.013s	sys 0.015s
I20260709 12:23:00.368521  4413 tablet_bootstrap.cc:492] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.376138  4413 tablet_bootstrap.cc:654] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.389523  4418 raft_consensus.cc:493] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.389798  4413 tablet_bootstrap.cc:492] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.390758  4413 ts_tablet_manager.cc:1403] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 12:23:00.390317  4418 raft_consensus.cc:515] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.394568  4413 raft_consensus.cc:359] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.395399  4413 raft_consensus.cc:385] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.395714  4413 raft_consensus.cc:740] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.396139  4418 leader_election.cc:290] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495), 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539)
I20260709 12:23:00.396756  4413 consensus_queue.cc:260] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.399663  4413 ts_tablet_manager.cc:1434] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:23:00.400940  4413 tablet_bootstrap.cc:492] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.421952  4413 tablet_bootstrap.cc:654] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:00.424903  4398 tablet.cc:2406] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:00.427605  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
W20260709 12:23:00.430290  4261 leader_election.cc:343] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.433390  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:00.434434  4223 raft_consensus.cc:2468] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:00.436743  4413 tablet_bootstrap.cc:492] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.436961  4264 leader_election.cc:304] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:00.437587  4413 ts_tablet_manager.cc:1403] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.003s
I20260709 12:23:00.438032  4418 raft_consensus.cc:2804] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.438555  4418 raft_consensus.cc:493] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.438931  4418 raft_consensus.cc:3060] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.441264  4413 raft_consensus.cc:359] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.442107  4413 raft_consensus.cc:385] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.442425  4413 raft_consensus.cc:740] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.443574  4413 consensus_queue.cc:260] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.446209  4413 ts_tablet_manager.cc:1434] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:23:00.447364  4413 tablet_bootstrap.cc:492] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.455318  4413 tablet_bootstrap.cc:654] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.453655  4418 raft_consensus.cc:515] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.456789  4414 tablet_bootstrap.cc:492] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.457376  4414 ts_tablet_manager.cc:1403] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.052s
I20260709 12:23:00.459527  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
W20260709 12:23:00.461077  4261 leader_election.cc:343] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.460465  4414 raft_consensus.cc:359] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.461777  4414 raft_consensus.cc:385] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.462121  4414 raft_consensus.cc:740] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.462459  4418 leader_election.cc:290] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Requested vote from peers 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495), 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539)
I20260709 12:23:00.462917  4414 consensus_queue.cc:260] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.464548  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:00.464974  4414 ts_tablet_manager.cc:1434] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:23:00.465505  4223 raft_consensus.cc:3060] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.466274  4414 tablet_bootstrap.cc:492] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.471977  4414 tablet_bootstrap.cc:654] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.473943  4412 tablet_bootstrap.cc:492] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:00.474472  4412 ts_tablet_manager.cc:1403] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.120s	user 0.038s	sys 0.044s
I20260709 12:23:00.477767  4223 raft_consensus.cc:2468] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:00.478637  4412 raft_consensus.cc:359] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.479813  4412 raft_consensus.cc:385] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.479861  4264 leader_election.cc:304] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:00.480422  4412 raft_consensus.cc:740] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.481226  4418 raft_consensus.cc:2804] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:00.481453  4412 consensus_queue.cc:260] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.482717  4418 raft_consensus.cc:697] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:00.484745  4412 ts_tablet_manager.cc:1434] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:23:00.484571  4418 consensus_queue.cc:237] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.488292  4414 tablet_bootstrap.cc:492] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.488899  4414 ts_tablet_manager.cc:1403] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.023s	user 0.003s	sys 0.011s
I20260709 12:23:00.491482  4412 tablet_bootstrap.cc:492] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:00.491972  4414 raft_consensus.cc:359] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.492923  4414 raft_consensus.cc:385] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.493229  4414 raft_consensus.cc:740] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.494221  4414 consensus_queue.cc:260] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.494412  4413 tablet_bootstrap.cc:492] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.496045  4413 ts_tablet_manager.cc:1403] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.022s
I20260709 12:23:00.496742  4414 ts_tablet_manager.cc:1434] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:23:00.497998  4414 tablet_bootstrap.cc:492] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.499473  4412 tablet_bootstrap.cc:654] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.503831  4413 raft_consensus.cc:359] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.505864  4414 tablet_bootstrap.cc:654] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:00.508874  4169 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:00.510138  4413 raft_consensus.cc:385] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.511121  4413 raft_consensus.cc:740] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.514263  4412 tablet_bootstrap.cc:492] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:00.513990  4413 consensus_queue.cc:260] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.514892  4412 ts_tablet_manager.cc:1403] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.012s
I20260709 12:23:00.516664  4413 ts_tablet_manager.cc:1434] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:23:00.517788  4413 tablet_bootstrap.cc:492] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.515044  4414 tablet_bootstrap.cc:492] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.519687  4414 ts_tablet_manager.cc:1403] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 12:23:00.520231  4119 catalog_manager.cc:5697] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.523190  4414 raft_consensus.cc:359] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.524070  4414 raft_consensus.cc:385] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.524407  4414 raft_consensus.cc:740] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.525239  4414 consensus_queue.cc:260] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.527254  4413 tablet_bootstrap.cc:654] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.527693  4414 ts_tablet_manager.cc:1434] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:23:00.528887  4414 tablet_bootstrap.cc:492] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.530572  4412 raft_consensus.cc:359] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.531461  4412 raft_consensus.cc:385] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.531807  4412 raft_consensus.cc:740] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.532661  4412 consensus_queue.cc:260] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.533609  4413 tablet_bootstrap.cc:492] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.534181  4413 ts_tablet_manager.cc:1403] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.003s
I20260709 12:23:00.534981  4412 ts_tablet_manager.cc:1434] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.003s
I20260709 12:23:00.536371  4412 tablet_bootstrap.cc:492] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:00.537348  4413 raft_consensus.cc:359] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.538139  4413 raft_consensus.cc:385] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.538422  4413 raft_consensus.cc:740] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.539403  4413 consensus_queue.cc:260] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.544028  4414 tablet_bootstrap.cc:654] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.544168  4412 tablet_bootstrap.cc:654] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.547523  4413 ts_tablet_manager.cc:1434] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:23:00.548763  4413 tablet_bootstrap.cc:492] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.555073  4412 tablet_bootstrap.cc:492] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:00.555789  4412 ts_tablet_manager.cc:1403] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:23:00.560459  4412 raft_consensus.cc:359] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.561265  4412 raft_consensus.cc:385] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.561586  4412 raft_consensus.cc:740] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.562616  4413 tablet_bootstrap.cc:654] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.562471  4412 consensus_queue.cc:260] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.567654  4412 ts_tablet_manager.cc:1434] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:23:00.568903  4412 tablet_bootstrap.cc:492] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:00.575565  4413 tablet_bootstrap.cc:492] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.576195  4413 ts_tablet_manager.cc:1403] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20260709 12:23:00.577361  4414 tablet_bootstrap.cc:492] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.578015  4414 ts_tablet_manager.cc:1403] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.007s
I20260709 12:23:00.579006  4413 raft_consensus.cc:359] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.579481  4412 tablet_bootstrap.cc:654] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.579671  4413 raft_consensus.cc:385] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.580181  4413 raft_consensus.cc:740] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.580900  4413 consensus_queue.cc:260] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.581719  4414 raft_consensus.cc:359] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.582693  4414 raft_consensus.cc:385] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.583056  4414 raft_consensus.cc:740] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.583498  4413 ts_tablet_manager.cc:1434] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:00.584599  4413 tablet_bootstrap.cc:492] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.584254  4414 consensus_queue.cc:260] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.587899  4414 ts_tablet_manager.cc:1434] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 12:23:00.589499  4414 tablet_bootstrap.cc:492] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.599305  4414 tablet_bootstrap.cc:654] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.601053  4413 tablet_bootstrap.cc:654] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.603668  4412 tablet_bootstrap.cc:492] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:00.604317  4412 ts_tablet_manager.cc:1403] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.010s
I20260709 12:23:00.605520  4414 tablet_bootstrap.cc:492] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.606153  4414 ts_tablet_manager.cc:1403] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:23:00.606715  4413 tablet_bootstrap.cc:492] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.607295  4413 ts_tablet_manager.cc:1403] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.001s
I20260709 12:23:00.607733  4412 raft_consensus.cc:359] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.608582  4412 raft_consensus.cc:385] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.608891  4412 raft_consensus.cc:740] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.609323  4414 raft_consensus.cc:359] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.609786  4412 consensus_queue.cc:260] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.610037  4413 raft_consensus.cc:359] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.610140  4414 raft_consensus.cc:385] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.610924  4413 raft_consensus.cc:385] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.611012  4414 raft_consensus.cc:740] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.611424  4413 raft_consensus.cc:740] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.612159  4414 consensus_queue.cc:260] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.612344  4413 consensus_queue.cc:260] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.614820  4414 ts_tablet_manager.cc:1434] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.002s
I20260709 12:23:00.617022  4414 tablet_bootstrap.cc:492] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.618479  4413 ts_tablet_manager.cc:1434] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:23:00.619616  4413 tablet_bootstrap.cc:492] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.620615  4412 ts_tablet_manager.cc:1434] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.007s
I20260709 12:23:00.624214  4420 raft_consensus.cc:493] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.624785  4420 raft_consensus.cc:515] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.625509  4412 tablet_bootstrap.cc:492] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:00.629065  4420 leader_election.cc:290] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.629336  4413 tablet_bootstrap.cc:654] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.624238  4414 tablet_bootstrap.cc:654] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.634630  4412 tablet_bootstrap.cc:654] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.650844  4419 raft_consensus.cc:493] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.651667  4419 raft_consensus.cc:515] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.658703  4419 leader_election.cc:290] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:00.724361  4419 raft_consensus.cc:493] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.725015  4419 raft_consensus.cc:515] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:00.728077  4418 raft_consensus.cc:493] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.728811  4418 raft_consensus.cc:515] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.730852  4419 leader_election.cc:290] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539)
I20260709 12:23:00.732378  4418 leader_election.cc:290] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.735584  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:00.736501  4372 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:00.739037  4420 raft_consensus.cc:493] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.739714  4420 raft_consensus.cc:515] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.745754  4420 leader_election.cc:290] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.747537  4261 leader_election.cc:304] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:00.749017  4418 raft_consensus.cc:2804] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.749564  4418 raft_consensus.cc:493] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.750073  4418 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.766103  4414 tablet_bootstrap.cc:492] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.767139  4414 ts_tablet_manager.cc:1403] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.150s	user 0.045s	sys 0.050s
I20260709 12:23:00.768886  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:00.769693  4223 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:00.771564  4414 raft_consensus.cc:359] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.772670  4414 raft_consensus.cc:385] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.773053  4414 raft_consensus.cc:740] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.774158  4414 consensus_queue.cc:260] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.775319  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3be6e6308145f7a32b2ede29daafa4" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:00.776440  4298 raft_consensus.cc:2468] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:00.777523  4414 ts_tablet_manager.cc:1434] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:00.779748  4414 tablet_bootstrap.cc:492] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.787021  4419 raft_consensus.cc:493] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.787799  4419 raft_consensus.cc:515] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.789269  4414 tablet_bootstrap.cc:654] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.789700  4429 raft_consensus.cc:493] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.790405  4429 raft_consensus.cc:515] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.793843  4420 raft_consensus.cc:493] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.794703  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
W20260709 12:23:00.796815  4264 leader_election.cc:343] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.798116  4429 leader_election.cc:290] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.794584  4420 raft_consensus.cc:515] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.805541  4414 tablet_bootstrap.cc:492] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.806360  4414 ts_tablet_manager.cc:1403] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.015s
I20260709 12:23:00.810242  4414 raft_consensus.cc:359] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.811206  4414 raft_consensus.cc:385] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.811558  4414 raft_consensus.cc:740] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.812405  4414 consensus_queue.cc:260] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.813575  4419 leader_election.cc:290] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:00.802240  4420 leader_election.cc:290] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.815986  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:00.816985  4298 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:00.819095  4414 ts_tablet_manager.cc:1434] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:23:00.820571  4414 tablet_bootstrap.cc:492] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:00.827847  4189 leader_election.cc:304] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:00.829253  4414 tablet_bootstrap.cc:654] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:00.829715  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:00.830776  4298 raft_consensus.cc:2468] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:00.831951  4420 raft_consensus.cc:2804] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.832616  4420 raft_consensus.cc:493] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.833112  4420 raft_consensus.cc:3060] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.839946  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:00.842991  4414 tablet_bootstrap.cc:492] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:00.842952  4189 leader_election.cc:304] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:00.843567  4414 ts_tablet_manager.cc:1403] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 12:23:00.847687  4414 raft_consensus.cc:359] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.848641  4414 raft_consensus.cc:385] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.848997  4414 raft_consensus.cc:740] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.849718  4414 consensus_queue.cc:260] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.852025  4414 ts_tablet_manager.cc:1434] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
W20260709 12:23:00.849952  4261 leader_election.cc:343] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.860388  4261 leader_election.cc:304] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:00.861368  4429 raft_consensus.cc:2749] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:00.862113  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:00.862993  4222 raft_consensus.cc:2393] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e5c80c10c344270b0d22ed21ccb1274 in current term 1: Already voted for candidate 53ebf9844ebb4152846b6e9fd1249a17 in this term.
I20260709 12:23:00.863634  4412 tablet_bootstrap.cc:492] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:00.864032  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:00.864285  4412 ts_tablet_manager.cc:1403] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.239s	user 0.089s	sys 0.137s
I20260709 12:23:00.864383  4413 tablet_bootstrap.cc:492] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:00.865104  4413 ts_tablet_manager.cc:1403] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.246s	user 0.054s	sys 0.161s
I20260709 12:23:00.865548  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:00.866181  4223 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
W20260709 12:23:00.867748  4338 leader_election.cc:343] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.867938  4412 raft_consensus.cc:359] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.868476  4338 leader_election.cc:304] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:00.868741  4412 raft_consensus.cc:385] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.869127  4412 raft_consensus.cc:740] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.869292  4420 raft_consensus.cc:515] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.869838  4419 raft_consensus.cc:2804] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.870718  4419 raft_consensus.cc:493] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.871345  4419 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.863703  4418 raft_consensus.cc:515] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.874136  4418 leader_election.cc:290] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.870002  4412 consensus_queue.cc:260] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.871016  4413 raft_consensus.cc:359] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.875495  4413 raft_consensus.cc:385] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:00.875447  4370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:00.875942  4413 raft_consensus.cc:740] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:00.875412  4420 leader_election.cc:290] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.876833  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:00.877245  4420 raft_consensus.cc:2804] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.877640  4369 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.878036  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:00.880353  4370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3be6e6308145f7a32b2ede29daafa4" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
W20260709 12:23:00.882207  4186 leader_election.cc:343] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.883486  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3be6e6308145f7a32b2ede29daafa4" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:00.885376  4369 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:00.876746  4413 consensus_queue.cc:260] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
W20260709 12:23:00.885821  4186 leader_election.cc:343] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:00.886855  4186 leader_election.cc:343] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:00.887760  4413 ts_tablet_manager.cc:1434] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 12:23:00.886861  4261 leader_election.cc:304] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:00.888906  4413 tablet_bootstrap.cc:492] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:00.889307  4418 raft_consensus.cc:2804] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:00.890728  4418 raft_consensus.cc:697] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:00.895224  4418 consensus_queue.cc:237] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.896720  4413 tablet_bootstrap.cc:654] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:00.891337  4189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:42909, user_credentials={real_user=slave}} blocked reactor thread for 48401.2us
I20260709 12:23:00.900094  4189 leader_election.cc:304] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:00.901041  4369 raft_consensus.cc:3060] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.877652  4420 raft_consensus.cc:493] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.901661  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:00.903357  4296 raft_consensus.cc:2468] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:00.902690  4420 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.905068  4295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3be6e6308145f7a32b2ede29daafa4" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:00.900115  4429 raft_consensus.cc:493] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.908222  4429 raft_consensus.cc:515] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.906013  4295 raft_consensus.cc:3060] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.901008  4297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:00.901028  4454 raft_consensus.cc:2804] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.919294  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:00.911207  4454 raft_consensus.cc:493] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.920085  4298 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:00.920509  4454 raft_consensus.cc:3060] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.929736  4412 ts_tablet_manager.cc:1434] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.065s	user 0.037s	sys 0.012s
I20260709 12:23:00.930506  4429 leader_election.cc:290] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:00.936973  4412 tablet_bootstrap.cc:492] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:00.926155  4338 leader_election.cc:304] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:00.942472  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:00.943485  4372 raft_consensus.cc:2468] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:00.945226  4261 leader_election.cc:304] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:00.946689  4429 raft_consensus.cc:2804] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.921800  4419 raft_consensus.cc:515] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:00.947309  4429 raft_consensus.cc:493] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.966019  4429 raft_consensus.cc:3060] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.974025  4371 raft_consensus.cc:2393] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50159f6246ed4d19bc4169a9c142b98d in current term 1: Already voted for candidate 3e5c80c10c344270b0d22ed21ccb1274 in this term.
I20260709 12:23:00.974972  4419 leader_election.cc:290] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:00.975903  4419 raft_consensus.cc:2804] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:00.976336  4419 raft_consensus.cc:493] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.976742  4419 raft_consensus.cc:3060] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:00.978423  4121 catalog_manager.cc:5697] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:00.982796  4458 raft_consensus.cc:493] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.983795  4458 raft_consensus.cc:515] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.021147  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:01.018802  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.022318  4223 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.026281  4412 tablet_bootstrap.cc:654] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:01.027133  4338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.23.217.66:42909, user_credentials={real_user=slave}} blocked reactor thread for 104512us
I20260709 12:23:01.028900  4413 tablet_bootstrap.cc:492] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:01.029451  4460 consensus_queue.cc:1048] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.030076  4297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:01.030323  4457 raft_consensus.cc:493] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.031764  4413 ts_tablet_manager.cc:1403] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.143s	user 0.019s	sys 0.043s
I20260709 12:23:01.034212  4297 raft_consensus.cc:2468] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:01.034987  4295 raft_consensus.cc:2468] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:01.033854  4418 consensus_queue.cc:1048] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:01.038033  4413 raft_consensus.cc:359] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.038872  4413 raft_consensus.cc:385] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.039148  4413 raft_consensus.cc:740] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.039835  4457 raft_consensus.cc:515] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.036430  4454 raft_consensus.cc:515] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.036720  4223 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:01.041220  4221 raft_consensus.cc:2468] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:01.031563  4458 leader_election.cc:290] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.043183  4413 consensus_queue.cc:260] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.041253  4429 raft_consensus.cc:515] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.048085  4189 leader_election.cc:304] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:01.049383  4458 raft_consensus.cc:2804] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.049873  4458 raft_consensus.cc:493] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.050331  4458 raft_consensus.cc:3060] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.051442  4419 raft_consensus.cc:515] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.053681  4419 leader_election.cc:290] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:01.047219  4420 raft_consensus.cc:515] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.056804  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.056969  4297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.057822  4221 raft_consensus.cc:2393] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e5c80c10c344270b0d22ed21ccb1274 in current term 1: Already voted for candidate 50159f6246ed4d19bc4169a9c142b98d in this term.
I20260709 12:23:01.057972  4297 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.059094  4412 tablet_bootstrap.cc:492] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:01.060400  4413 ts_tablet_manager.cc:1434] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.004s
I20260709 12:23:01.059738  4412 ts_tablet_manager.cc:1403] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.123s	user 0.015s	sys 0.006s
I20260709 12:23:01.065491  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.062851  4454 leader_election.cc:290] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.066169  4221 raft_consensus.cc:3060] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.067946  4420 leader_election.cc:290] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.067255  4457 leader_election.cc:290] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.071605  4338 leader_election.cc:304] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274; no voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.073177  4412 raft_consensus.cc:359] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.074040  4412 raft_consensus.cc:385] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.074193  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.074396  4412 raft_consensus.cc:740] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.076023  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.076277  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.076782  4296 raft_consensus.cc:3060] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.076469  4412 consensus_queue.cc:260] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.077101  4223 raft_consensus.cc:3060] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.076376  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:01.079054  4412 ts_tablet_manager.cc:1434] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 12:23:01.076695  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.083958  4419 raft_consensus.cc:2749] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.085650  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:01.085791  4189 leader_election.cc:304] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:01.086544  4298 raft_consensus.cc:2393] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50159f6246ed4d19bc4169a9c142b98d in current term 1: Already voted for candidate 53ebf9844ebb4152846b6e9fd1249a17 in this term.
I20260709 12:23:01.087639  4429 leader_election.cc:290] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.088200  4223 raft_consensus.cc:2468] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:01.090852  4295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.091569  4295 raft_consensus.cc:3060] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.095391  4463 raft_consensus.cc:2804] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.095556  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:01.096064  4463 raft_consensus.cc:697] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.096442  4371 raft_consensus.cc:2468] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:01.097016  4463 consensus_queue.cc:237] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.098389  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.103925  4412 tablet_bootstrap.cc:492] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:01.104030  4371 raft_consensus.cc:2393] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 50159f6246ed4d19bc4169a9c142b98d in current term 1: Already voted for candidate 3e5c80c10c344270b0d22ed21ccb1274 in this term.
I20260709 12:23:01.109058  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.112623  4372 raft_consensus.cc:3060] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:01.105481  4186 leader_election.cc:343] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.114017  4297 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
W20260709 12:23:01.118207  4186 leader_election.cc:343] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.118629  4295 raft_consensus.cc:2468] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:01.119072  4412 tablet_bootstrap.cc:654] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.119184  4186 leader_election.cc:304] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.119778  4189 leader_election.cc:304] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:01.120460  4338 leader_election.cc:304] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:01.122736  4457 raft_consensus.cc:2804] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.123251  4457 raft_consensus.cc:697] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.123667  4338 leader_election.cc:304] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:01.124446  4457 consensus_queue.cc:237] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.126834  4461 raft_consensus.cc:2804] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.127540  4461 raft_consensus.cc:697] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:01.128458  4461 consensus_queue.cc:237] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.115768  4296 raft_consensus.cc:2468] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:01.132936  4186 leader_election.cc:304] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d; no voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.133910  4454 raft_consensus.cc:2749] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.120618  4458 raft_consensus.cc:515] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.123242  4419 raft_consensus.cc:2804] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.144284  4419 raft_consensus.cc:697] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:01.145682  4419 consensus_queue.cc:237] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.159917  4458 leader_election.cc:290] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.162197  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.162401  4221 raft_consensus.cc:2468] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:01.162951  4296 raft_consensus.cc:3060] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.162707  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.163834  4264 leader_election.cc:304] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
W20260709 12:23:01.165112  4186 leader_election.cc:343] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:01.166795  4418 raft_consensus.cc:2804] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.166962  4412 tablet_bootstrap.cc:492] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:01.167821  4412 ts_tablet_manager.cc:1403] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.010s
I20260709 12:23:01.167435  4418 raft_consensus.cc:697] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:01.162989  4420 raft_consensus.cc:2764] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:23:01.169226  4418 consensus_queue.cc:237] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.186439  4296 raft_consensus.cc:2468] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:01.186728  4455 raft_consensus.cc:493] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.187505  4455 raft_consensus.cc:515] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.189373  4189 leader_election.cc:304] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:01.190332  4420 raft_consensus.cc:2804] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.190932  4420 raft_consensus.cc:697] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.191646  4455 leader_election.cc:290] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:01.191823  4420 consensus_queue.cc:237] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.193419  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:01.194142  4222 raft_consensus.cc:2468] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:01.195407  4338 leader_election.cc:304] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:01.195502  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:01.196285  4455 raft_consensus.cc:2804] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:01.196751  4455 raft_consensus.cc:493] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.196314  4296 raft_consensus.cc:2468] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:01.197111  4455 raft_consensus.cc:3060] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.203949  4412 raft_consensus.cc:359] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.204610  4412 raft_consensus.cc:385] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.204910  4412 raft_consensus.cc:740] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.205598  4412 consensus_queue.cc:260] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.209724  4120 catalog_manager.cc:5697] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.212203  4372 raft_consensus.cc:2468] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:01.214040  4412 ts_tablet_manager.cc:1434] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 12:23:01.215308  4412 tablet_bootstrap.cc:492] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:01.215097  4121 catalog_manager.cc:5697] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 0 to 1, leader changed from <none> to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 1 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.217029  4121 catalog_manager.cc:5697] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 0 to 1, leader changed from <none> to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 1 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.225528  4455 raft_consensus.cc:515] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.229415  4412 tablet_bootstrap.cc:654] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.229763  4455 leader_election.cc:290] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:01.231570  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.232326  4296 raft_consensus.cc:3060] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.229885  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.234948  4222 raft_consensus.cc:3060] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.237668  4119 catalog_manager.cc:5697] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:01.239557  4119 catalog_manager.cc:5697] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:01.244400  4222 raft_consensus.cc:2468] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:01.244747  4412 tablet_bootstrap.cc:492] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:01.245393  4412 ts_tablet_manager.cc:1403] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.001s
I20260709 12:23:01.247061  4338 leader_election.cc:304] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:01.248234  4419 raft_consensus.cc:2804] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.248487  4412 raft_consensus.cc:359] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.249055  4419 raft_consensus.cc:697] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:01.249338  4412 raft_consensus.cc:385] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.249545  4296 raft_consensus.cc:2468] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:01.249867  4412 raft_consensus.cc:740] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.250511  4419 consensus_queue.cc:237] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.250880  4412 consensus_queue.cc:260] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.253435  4412 ts_tablet_manager.cc:1434] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.008s
I20260709 12:23:01.260725  4120 catalog_manager.cc:5697] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 0 to 1, leader changed from <none> to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 1 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.273926  4119 catalog_manager.cc:5697] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:01.317248  4418 consensus_queue.cc:1048] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.330691  4418 consensus_queue.cc:1048] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.515637  4420 consensus_queue.cc:1048] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.522225  4419 consensus_queue.cc:1048] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.525017  4419 raft_consensus.cc:493] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.525568  4419 raft_consensus.cc:515] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.528429  4419 leader_election.cc:290] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:01.528785  4372 raft_consensus.cc:3060] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.530642  4295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3be6e6308145f7a32b2ede29daafa4" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:01.532881  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3be6e6308145f7a32b2ede29daafa4" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:01.534480  4338 leader_election.cc:304] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274; no voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.540139  4419 raft_consensus.cc:2749] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.543076  4420 consensus_queue.cc:1048] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.544162  4419 consensus_queue.cc:1048] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.556879  4419 raft_consensus.cc:493] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.557695  4419 raft_consensus.cc:515] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.559866  4419 leader_election.cc:290] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:01.561702  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:01.562944  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:01.563784  4296 raft_consensus.cc:2393] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e5c80c10c344270b0d22ed21ccb1274 in current term 1: Already voted for candidate 50159f6246ed4d19bc4169a9c142b98d in this term.
I20260709 12:23:01.565954  4338 leader_election.cc:304] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274; no voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.567360  4461 raft_consensus.cc:3060] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.575781  4461 raft_consensus.cc:2749] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.576639  4420 consensus_queue.cc:1048] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.579071  4295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
mode: GRACEFUL
new_leader_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:41048
I20260709 12:23:01.579810  4295 raft_consensus.cc:606] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Received request to transfer leadership to TS 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:01.586253  4295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f936823aef084ec084d006f0af36a144"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
mode: GRACEFUL
new_leader_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:41048
I20260709 12:23:01.586881  4295 raft_consensus.cc:606] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Received request to transfer leadership to TS 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:01.594307  4420 consensus_queue.cc:1048] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.594427  4418 raft_consensus.cc:993] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17: : Instructing follower 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:01.595302  4418 raft_consensus.cc:1081] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Signalling peer 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:01.597137  4221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:35618
I20260709 12:23:01.597784  4221 raft_consensus.cc:493] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:01.598188  4221 raft_consensus.cc:3060] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.607154  4221 raft_consensus.cc:515] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:01.610351  4221 leader_election.cc:290] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.611335  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.611985  4296 raft_consensus.cc:3055] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:01.612212  4296 raft_consensus.cc:740] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:01.613122  4372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
mode: GRACEFUL
new_leader_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:54400
I20260709 12:23:01.613070  4296 consensus_queue.cc:260] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:01.614281  4296 raft_consensus.cc:3060] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.614197  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a23c16600842d39941c8d6416ebf54" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.614681  4372 raft_consensus.cc:606] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Received request to transfer leadership to TS 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:01.615044  4369 raft_consensus.cc:3060] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.622426  4296 raft_consensus.cc:2468] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:01.623193  4369 raft_consensus.cc:2468] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:01.623979  4189 leader_election.cc:304] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:01.624981  4420 raft_consensus.cc:2804] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:01.625516  4420 raft_consensus.cc:697] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.625591  4369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
mode: GRACEFUL
new_leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:54400
I20260709 12:23:01.626309  4369 raft_consensus.cc:606] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Received request to transfer leadership to TS 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.626384  4420 consensus_queue.cc:237] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } }
I20260709 12:23:01.634426  4121 catalog_manager.cc:5697] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 1 to 2, leader changed from 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66) to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 2 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:01.637113  4419 consensus_queue.cc:1048] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.641620  4479 consensus_queue.cc:1048] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.649466  4223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
mode: GRACEFUL
new_leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:35626
I20260709 12:23:01.650095  4223 raft_consensus.cc:606] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Received request to transfer leadership to TS 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.650607  4419 consensus_queue.cc:1048] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.657032  4463 consensus_queue.cc:1048] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.658635  4418 consensus_queue.cc:1048] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.665560  4455 raft_consensus.cc:993] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274: : Instructing follower 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:01.666138  4455 raft_consensus.cc:1081] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Signalling peer 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:01.669039  4461 raft_consensus.cc:493] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.669516  4461 raft_consensus.cc:515] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.669799  4222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:35624
I20260709 12:23:01.671373  4222 raft_consensus.cc:493] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:01.671859  4222 raft_consensus.cc:3060] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.674435  4463 raft_consensus.cc:993] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d: : Instructing follower 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:01.675233  4463 raft_consensus.cc:1081] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Signalling peer 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:01.679236  4419 consensus_queue.cc:1048] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.682255  4298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:41034
I20260709 12:23:01.683178  4298 raft_consensus.cc:493] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:01.681324  4222 raft_consensus.cc:515] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.683763  4298 raft_consensus.cc:3060] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.685757  4222 leader_election.cc:290] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.687664  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:01.688722  4461 leader_election.cc:290] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:01.694018  4293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:01.699020  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.700446  4338 leader_election.cc:304] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274; no voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.700707  4370 raft_consensus.cc:3060] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.703900  4371 raft_consensus.cc:3055] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:01.704342  4371 raft_consensus.cc:740] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:01.705188  4371 consensus_queue.cc:260] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.705835  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7cac0b3ada94e2ab2953640133556f7" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.711437  4296 raft_consensus.cc:3060] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.706383  4371 raft_consensus.cc:3060] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.710824  4419 raft_consensus.cc:2749] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:01.711774  4420 consensus_queue.cc:1048] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:01.714910  4298 raft_consensus.cc:515] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.718139  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.718801  4221 raft_consensus.cc:3055] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:01.719099  4221 raft_consensus.cc:740] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.720175  4297 raft_consensus.cc:1240] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Rejecting Update request from peer 50159f6246ed4d19bc4169a9c142b98d for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:01.720347  4296 raft_consensus.cc:2468] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:01.719931  4221 consensus_queue.cc:260] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.709625  4372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
mode: GRACEFUL
new_leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:54400
I20260709 12:23:01.721279  4221 raft_consensus.cc:3060] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.721872  4189 leader_election.cc:304] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:01.722903  4420 raft_consensus.cc:2804] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:01.723003  4461 raft_consensus.cc:993] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274: : Instructing follower 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:01.723531  4420 raft_consensus.cc:697] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.723704  4461 raft_consensus.cc:1081] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Signalling peer 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:01.725623  4372 raft_consensus.cc:606] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Received request to transfer leadership to TS 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.724517  4420 consensus_queue.cc:237] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.731630  4371 raft_consensus.cc:2468] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:01.735203  4297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:41046
I20260709 12:23:01.735445  4298 leader_election.cc:290] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.737505  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ebc79e324843fc9926cf5699e20916" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.738101  4297 raft_consensus.cc:493] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:01.738670  4297 raft_consensus.cc:3060] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.739544  4121 catalog_manager.cc:5697] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 1 to 2, leader changed from 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67) to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 2 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.740998  4455 consensus_queue.cc:1048] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:01.743753  4221 raft_consensus.cc:2468] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:01.748251  4264 leader_election.cc:304] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:01.749185  4223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
mode: GRACEFUL
new_leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:35626
I20260709 12:23:01.749815  4223 raft_consensus.cc:606] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Received request to transfer leadership to TS 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:01.750100  4418 raft_consensus.cc:2804] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:01.750756  4418 raft_consensus.cc:697] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:01.751533  4429 raft_consensus.cc:993] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17: : Instructing follower 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:01.752105  4429 raft_consensus.cc:1081] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Signalling peer 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:01.751601  4418 consensus_queue.cc:237] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.760351  4297 raft_consensus.cc:515] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.762234  4298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
mode: GRACEFUL
new_leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:41048
I20260709 12:23:01.763069  4298 raft_consensus.cc:606] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Received request to transfer leadership to TS 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:01.763621  4297 leader_election.cc:290] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.764161  4223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f936823aef084ec084d006f0af36a144"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:35618
I20260709 12:23:01.764961  4370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.765064  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51aba7e903494170beedc9ed8d56e026" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.766072  4370 raft_consensus.cc:3055] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:01.766510  4370 raft_consensus.cc:740] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:01.767422  4370 consensus_queue.cc:260] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.764907  4223 raft_consensus.cc:493] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:01.768630  4223 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.768733  4370 raft_consensus.cc:3060] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.778191  4370 raft_consensus.cc:2468] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:01.778223  4223 raft_consensus.cc:515] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.779763  4261 leader_election.cc:304] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:01.780903  4418 raft_consensus.cc:2804] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:01.781414  4418 raft_consensus.cc:697] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:01.782300  4418 consensus_queue.cc:237] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.782907  4223 leader_election.cc:290] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.784741  4121 catalog_manager.cc:5697] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 1 to 2, leader changed from 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65) to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.785619  4297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:01.786324  4297 raft_consensus.cc:3055] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:01.786716  4297 raft_consensus.cc:740] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:01.787595  4297 consensus_queue.cc:260] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.787756  4370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.788619  4370 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.788800  4297 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.795621  4297 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:01.795954  4370 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:01.796761  4189 leader_election.cc:304] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:01.797747  4420 raft_consensus.cc:2804] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:01.798295  4420 raft_consensus.cc:697] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.799147  4420 consensus_queue.cc:237] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.805469  4121 catalog_manager.cc:5697] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 1 to 2, leader changed from 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67) to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.807154  4120 catalog_manager.cc:5697] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 1 to 2, leader changed from 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66) to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 2 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:01.984575  4463 raft_consensus.cc:993] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d: : Instructing follower 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:01.985100  4420 raft_consensus.cc:1081] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Signalling peer 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:01.987006  4297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:41034
I20260709 12:23:01.987537  4297 raft_consensus.cc:493] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:01.987854  4297 raft_consensus.cc:3060] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.993202  4297 raft_consensus.cc:515] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.995359  4297 leader_election.cc:290] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:01.996289  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:01.996942  4223 raft_consensus.cc:3055] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:01.997294  4223 raft_consensus.cc:740] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:01.997289  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "291c358aca9c4f399b45bfc8f1884c32" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:01.998318  4371 raft_consensus.cc:3060] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:01.998598  4223 consensus_queue.cc:260] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:01.999984  4223 raft_consensus.cc:3060] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.002727  4461 raft_consensus.cc:993] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274: : Instructing follower 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:02.003341  4455 raft_consensus.cc:1081] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Signalling peer 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:02.005041  4297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:41046
I20260709 12:23:02.005555  4297 raft_consensus.cc:493] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:02.005906  4297 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.005807  4371 raft_consensus.cc:2468] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:02.007302  4223 raft_consensus.cc:2468] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:02.007769  4261 leader_election.cc:304] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:02.008702  4418 raft_consensus.cc:2804] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:02.009114  4418 raft_consensus.cc:697] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:02.009955  4418 consensus_queue.cc:237] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:02.013124  4297 raft_consensus.cc:515] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:02.015415  4297 leader_election.cc:290] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:02.016016  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:02.016652  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:02.016696  4223 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.017449  4371 raft_consensus.cc:3055] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:02.017869  4371 raft_consensus.cc:740] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:02.017297  4120 catalog_manager.cc:5697] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 1 to 2, leader changed from 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65) to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:02.018828  4371 consensus_queue.cc:260] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:02.020251  4371 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.025103  4223 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:02.026432  4264 leader_election.cc:304] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:02.027307  4418 raft_consensus.cc:2804] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:02.027882  4418 raft_consensus.cc:697] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:02.028276  4371 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:02.028631  4418 consensus_queue.cc:237] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:02.035290  4120 catalog_manager.cc:5697] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 1 to 2, leader changed from 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67) to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:02.106413  4371 raft_consensus.cc:1275] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.108022  4420 consensus_queue.cc:1048] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.117192  4297 raft_consensus.cc:1275] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.118914  4463 consensus_queue.cc:1048] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:02.211314  4297 raft_consensus.cc:1275] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.212738  4463 consensus_queue.cc:1048] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.218276  4370 raft_consensus.cc:1275] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.219314  4479 raft_consensus.cc:993] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17: : Instructing follower 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:02.219961  4429 raft_consensus.cc:1081] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Signalling peer 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:02.220727  4479 consensus_queue.cc:1048] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.223702  4370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:54376
I20260709 12:23:02.224267  4370 raft_consensus.cc:493] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:02.224635  4370 raft_consensus.cc:3060] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.225186  4372 raft_consensus.cc:1275] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.229856  4420 consensus_queue.cc:1048] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.235052  4297 raft_consensus.cc:1275] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.235596  4370 raft_consensus.cc:515] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:02.236769  4420 consensus_queue.cc:1048] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.237668  4223 raft_consensus.cc:3060] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.237933  4370 leader_election.cc:290] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:02.240744  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:02.240888  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7e9dd20dd6474e9fdfbd4a4b4fbb73" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:02.241458  4222 raft_consensus.cc:3060] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.241573  4298 raft_consensus.cc:3055] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:02.242123  4298 raft_consensus.cc:740] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:02.243067  4298 consensus_queue.cc:260] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:02.244364  4298 raft_consensus.cc:3060] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.251853  4223 raft_consensus.cc:1275] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.253320  4298 raft_consensus.cc:2468] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 2.
I20260709 12:23:02.253561  4371 raft_consensus.cc:1275] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.254618  4222 raft_consensus.cc:2468] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 2.
I20260709 12:23:02.257135  4418 consensus_queue.cc:1048] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.257347  4338 leader_election.cc:304] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:02.258172  4455 raft_consensus.cc:2804] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:02.257405  4420 consensus_queue.cc:1048] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:23:02.259137  4455 raft_consensus.cc:697] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:02.260319  4455 consensus_queue.cc:237] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:02.276566  4121 catalog_manager.cc:5697] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 1 to 2, leader changed from 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66) to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:02.301290  4371 raft_consensus.cc:3060] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:02.306584  4371 raft_consensus.cc:1275] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.307950  4418 consensus_queue.cc:1048] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.316454  4223 raft_consensus.cc:1275] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.317951  4418 consensus_queue.cc:1048] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.495481  4223 raft_consensus.cc:1275] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.496986  4418 consensus_queue.cc:1048] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.507804  4371 raft_consensus.cc:1275] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.509423  4479 consensus_queue.cc:1048] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.619653  4371 raft_consensus.cc:1275] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.621583  4479 consensus_queue.cc:1048] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.622627  4223 raft_consensus.cc:1275] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.624138  4479 consensus_queue.cc:1048] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.671382  4223 raft_consensus.cc:1275] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.672734  4455 consensus_queue.cc:1048] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.682868  4298 raft_consensus.cc:1275] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:02.684684  4461 consensus_queue.cc:1048] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:04.807089  4120 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40272:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:04.809798  4120 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:04.873734  4211 tablet_service.cc:1511] Processing CreateTablet for tablet a960b1c7238842a5b31e0f5d208287cb (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:04.876058  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a960b1c7238842a5b31e0f5d208287cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.877133  4355 tablet_service.cc:1511] Processing CreateTablet for tablet a960b1c7238842a5b31e0f5d208287cb (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:04.879313  4355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a960b1c7238842a5b31e0f5d208287cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.879560  4280 tablet_service.cc:1511] Processing CreateTablet for tablet a960b1c7238842a5b31e0f5d208287cb (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:04.885181  4280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a960b1c7238842a5b31e0f5d208287cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.891469  4209 tablet_service.cc:1511] Processing CreateTablet for tablet dcb163fdfe074544b7eb876f26ec6570 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:04.893143  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb163fdfe074544b7eb876f26ec6570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.895129  4359 tablet_service.cc:1511] Processing CreateTablet for tablet dcb163fdfe074544b7eb876f26ec6570 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:04.896642  4359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb163fdfe074544b7eb876f26ec6570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.896725  4283 tablet_service.cc:1511] Processing CreateTablet for tablet dcb163fdfe074544b7eb876f26ec6570 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:04.898435  4283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb163fdfe074544b7eb876f26ec6570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.906864  4207 tablet_service.cc:1511] Processing CreateTablet for tablet d385031a46bf491cabbcc60263ac6e9a (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:04.908725  4207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d385031a46bf491cabbcc60263ac6e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.908500  4285 tablet_service.cc:1511] Processing CreateTablet for tablet d385031a46bf491cabbcc60263ac6e9a (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:04.910166  4285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d385031a46bf491cabbcc60263ac6e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.947237  4357 tablet_service.cc:1511] Processing CreateTablet for tablet d385031a46bf491cabbcc60263ac6e9a (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:04.948848  4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d385031a46bf491cabbcc60263ac6e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.956327  4208 tablet_service.cc:1511] Processing CreateTablet for tablet 97ecb652aac54a2fbb426cd63803e74a (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:04.957870  4208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ecb652aac54a2fbb426cd63803e74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.957813  4281 tablet_service.cc:1511] Processing CreateTablet for tablet 97ecb652aac54a2fbb426cd63803e74a (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:04.960180  4281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ecb652aac54a2fbb426cd63803e74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.964721  4521 tablet_bootstrap.cc:492] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:04.987306  4359 tablet_service.cc:1511] Processing CreateTablet for tablet 97ecb652aac54a2fbb426cd63803e74a (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:04.988715  4359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ecb652aac54a2fbb426cd63803e74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:04.990330  4207 tablet_service.cc:1511] Processing CreateTablet for tablet fc68d9f3528b479baf8a639760570f70 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:04.991956  4207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc68d9f3528b479baf8a639760570f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.002912  4522 tablet_bootstrap.cc:492] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.008436  4281 tablet_service.cc:1511] Processing CreateTablet for tablet fc68d9f3528b479baf8a639760570f70 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:05.012248  4523 tablet_bootstrap.cc:492] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.011993  4355 tablet_service.cc:1511] Processing CreateTablet for tablet fc68d9f3528b479baf8a639760570f70 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:05.016638  4283 tablet_service.cc:1511] Processing CreateTablet for tablet 2263f966bfcd4876ad9943d998fab653 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:05.019104  4521 tablet_bootstrap.cc:654] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.018299  4360 tablet_service.cc:1511] Processing CreateTablet for tablet 2263f966bfcd4876ad9943d998fab653 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:05.020051  4355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc68d9f3528b479baf8a639760570f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.020804  4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2263f966bfcd4876ad9943d998fab653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.022228  4208 tablet_service.cc:1511] Processing CreateTablet for tablet c40760ebaa0e422c881e8efcbcbe83d4 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:05.023351  4281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc68d9f3528b479baf8a639760570f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.027894  4283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2263f966bfcd4876ad9943d998fab653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.027552  4280 tablet_service.cc:1511] Processing CreateTablet for tablet c40760ebaa0e422c881e8efcbcbe83d4 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:05.028954  4280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40760ebaa0e422c881e8efcbcbe83d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.030974  4208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40760ebaa0e422c881e8efcbcbe83d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.033389  4211 tablet_service.cc:1511] Processing CreateTablet for tablet 2263f966bfcd4876ad9943d998fab653 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:05.033811  4357 tablet_service.cc:1511] Processing CreateTablet for tablet c40760ebaa0e422c881e8efcbcbe83d4 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:05.040731  4522 tablet_bootstrap.cc:654] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.059141  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2263f966bfcd4876ad9943d998fab653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.059361  4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40760ebaa0e422c881e8efcbcbe83d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.045981  4523 tablet_bootstrap.cc:654] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.064935  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 752c55dfc1984731b3f2604f63ccae71 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:05.075343  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 752c55dfc1984731b3f2604f63ccae71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.075537  4285 tablet_service.cc:1511] Processing CreateTablet for tablet 752c55dfc1984731b3f2604f63ccae71 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:05.077018  4285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 752c55dfc1984731b3f2604f63ccae71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.081115  4213 tablet_service.cc:1511] Processing CreateTablet for tablet 3745b69c0304470b91ff96889e8485f2 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:05.082624  4213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3745b69c0304470b91ff96889e8485f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.089061  4287 tablet_service.cc:1511] Processing CreateTablet for tablet 3745b69c0304470b91ff96889e8485f2 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:05.089205  4360 tablet_service.cc:1511] Processing CreateTablet for tablet 752c55dfc1984731b3f2604f63ccae71 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:05.090667  4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 752c55dfc1984731b3f2604f63ccae71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.090714  4287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3745b69c0304470b91ff96889e8485f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.096122  4522 tablet_bootstrap.cc:492] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.108358  4521 tablet_bootstrap.cc:492] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.106925  4522 ts_tablet_manager.cc:1403] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.124s	user 0.002s	sys 0.012s
I20260709 12:23:05.117240  4521 ts_tablet_manager.cc:1403] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.153s	user 0.015s	sys 0.011s
I20260709 12:23:05.120390  4521 raft_consensus.cc:359] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.120224  4522 raft_consensus.cc:359] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.121501  4522 raft_consensus.cc:385] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.121879  4522 raft_consensus.cc:740] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.124665  4521 raft_consensus.cc:385] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.122747  4522 consensus_queue.cc:260] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.125139  4521 raft_consensus.cc:740] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.127242  4523 tablet_bootstrap.cc:492] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.128005  4523 ts_tablet_manager.cc:1403] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.116s	user 0.012s	sys 0.002s
I20260709 12:23:05.127457  4521 consensus_queue.cc:260] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.132032  4523 raft_consensus.cc:359] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.132925  4523 raft_consensus.cc:385] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.133273  4523 raft_consensus.cc:740] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.134116  4523 consensus_queue.cc:260] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.134750  4359 tablet_service.cc:1511] Processing CreateTablet for tablet 3745b69c0304470b91ff96889e8485f2 (DEFAULT_TABLE table=soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:05.137506  4523 ts_tablet_manager.cc:1434] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:23:05.137717  4522 ts_tablet_manager.cc:1434] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:23:05.138773  4522 tablet_bootstrap.cc:492] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.146457  4359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3745b69c0304470b91ff96889e8485f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:05.148267  4523 tablet_bootstrap.cc:492] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.152086  4521 ts_tablet_manager.cc:1434] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.002s
I20260709 12:23:05.153247  4521 tablet_bootstrap.cc:492] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.154420  4522 tablet_bootstrap.cc:654] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.156349  4523 tablet_bootstrap.cc:654] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.162567  4521 tablet_bootstrap.cc:654] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.169914  4506 raft_consensus.cc:493] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.170745  4506 raft_consensus.cc:515] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.174561  4522 tablet_bootstrap.cc:492] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.175462  4522 ts_tablet_manager.cc:1403] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.037s	user 0.003s	sys 0.014s
I20260709 12:23:05.178247  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:05.179421  4298 raft_consensus.cc:2468] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:05.181643  4523 tablet_bootstrap.cc:492] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.182389  4523 ts_tablet_manager.cc:1403] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.012s
I20260709 12:23:05.181885  4338 leader_election.cc:304] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.184222  4522 raft_consensus.cc:359] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.175489  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:05.185127  4522 raft_consensus.cc:385] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.185532  4522 raft_consensus.cc:740] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.185658  4223 raft_consensus.cc:2468] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:05.185940  4523 raft_consensus.cc:359] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.186785  4523 raft_consensus.cc:385] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.186460  4522 consensus_queue.cc:260] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.187124  4523 raft_consensus.cc:740] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.187947  4523 consensus_queue.cc:260] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.190097  4522 ts_tablet_manager.cc:1434] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:23:05.190539  4523 ts_tablet_manager.cc:1434] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:05.191637  4523 tablet_bootstrap.cc:492] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.191344  4522 tablet_bootstrap.cc:492] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.188295  4506 leader_election.cc:290] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:05.199009  4506 raft_consensus.cc:2804] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.199715  4506 raft_consensus.cc:493] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.200146  4506 raft_consensus.cc:3060] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.202203  4522 tablet_bootstrap.cc:654] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.212388  4523 tablet_bootstrap.cc:654] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.221688  4506 raft_consensus.cc:515] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.226835  4523 tablet_bootstrap.cc:492] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.227591  4523 ts_tablet_manager.cc:1403] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.004s
I20260709 12:23:05.227592  4521 tablet_bootstrap.cc:492] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.228044  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:05.227592  4522 tablet_bootstrap.cc:492] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.228799  4223 raft_consensus.cc:3060] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.229084  4522 ts_tablet_manager.cc:1403] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.009s
I20260709 12:23:05.228338  4521 ts_tablet_manager.cc:1403] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.075s	user 0.026s	sys 0.039s
I20260709 12:23:05.231353  4523 raft_consensus.cc:359] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.232250  4523 raft_consensus.cc:385] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.233039  4523 raft_consensus.cc:740] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.232187  4522 raft_consensus.cc:359] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.233727  4522 raft_consensus.cc:385] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.233541  4521 raft_consensus.cc:359] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.234134  4522 raft_consensus.cc:740] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.234479  4521 raft_consensus.cc:385] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.234035  4523 consensus_queue.cc:260] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.235054  4521 raft_consensus.cc:740] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.235469  4522 consensus_queue.cc:260] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.236150  4521 consensus_queue.cc:260] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.237128  4523 ts_tablet_manager.cc:1434] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:05.238381  4523 tablet_bootstrap.cc:492] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.238385  4522 ts_tablet_manager.cc:1434] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:23:05.238744  4521 ts_tablet_manager.cc:1434] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:23:05.240368  4521 tablet_bootstrap.cc:492] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.247421  4521 tablet_bootstrap.cc:654] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.249838  4522 tablet_bootstrap.cc:492] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.251950  4523 tablet_bootstrap.cc:654] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.266762  4522 tablet_bootstrap.cc:654] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.272622  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:05.272986  4223 raft_consensus.cc:2468] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:05.273351  4298 raft_consensus.cc:3060] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.274322  4338 leader_election.cc:304] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:05.275516  4530 raft_consensus.cc:2804] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.271670  4506 leader_election.cc:290] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:05.284029  4523 tablet_bootstrap.cc:492] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.287036  4523 ts_tablet_manager.cc:1403] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.013s
I20260709 12:23:05.285806  4298 raft_consensus.cc:2468] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:05.290328  4521 tablet_bootstrap.cc:492] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.291074  4521 ts_tablet_manager.cc:1403] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.021s
I20260709 12:23:05.291433  4530 raft_consensus.cc:697] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:05.291457  4523 raft_consensus.cc:359] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.292531  4523 raft_consensus.cc:385] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.292917  4523 raft_consensus.cc:740] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.292907  4530 consensus_queue.cc:237] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.294281  4523 consensus_queue.cc:260] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.294776  4521 raft_consensus.cc:359] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.299333  4522 tablet_bootstrap.cc:492] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.300189  4522 ts_tablet_manager.cc:1403] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.011s
I20260709 12:23:05.303218  4521 raft_consensus.cc:385] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.304462  4523 ts_tablet_manager.cc:1434] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:23:05.307345  4521 raft_consensus.cc:740] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.308403  4521 consensus_queue.cc:260] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.306350  4522 raft_consensus.cc:359] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.310477  4522 raft_consensus.cc:385] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.310822  4521 ts_tablet_manager.cc:1434] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.003s
I20260709 12:23:05.310920  4522 raft_consensus.cc:740] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.308423  4523 tablet_bootstrap.cc:492] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.312248  4522 consensus_queue.cc:260] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.313633  4118 catalog_manager.cc:5697] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:05.315222  4522 ts_tablet_manager.cc:1434] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:23:05.316458  4522 tablet_bootstrap.cc:492] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.316748  4521 tablet_bootstrap.cc:492] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.323989  4522 tablet_bootstrap.cc:654] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.324877  4521 tablet_bootstrap.cc:654] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.332747  4523 tablet_bootstrap.cc:654] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.340764  4521 tablet_bootstrap.cc:492] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.341084  4522 tablet_bootstrap.cc:492] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.341300  4521 ts_tablet_manager.cc:1403] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.005s
I20260709 12:23:05.341759  4522 ts_tablet_manager.cc:1403] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.007s
I20260709 12:23:05.344820  4523 tablet_bootstrap.cc:492] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.344702  4521 raft_consensus.cc:359] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.345511  4523 ts_tablet_manager.cc:1403] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.005s
I20260709 12:23:05.345664  4521 raft_consensus.cc:385] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.345333  4522 raft_consensus.cc:359] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.346124  4521 raft_consensus.cc:740] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.346300  4522 raft_consensus.cc:385] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.347324  4521 consensus_queue.cc:260] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.347792  4522 raft_consensus.cc:740] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.348809  4522 consensus_queue.cc:260] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.348929  4523 raft_consensus.cc:359] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.351851  4522 ts_tablet_manager.cc:1434] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:23:05.349782  4523 raft_consensus.cc:385] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.352212  4521 ts_tablet_manager.cc:1434] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:23:05.352506  4523 raft_consensus.cc:740] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.353123  4522 tablet_bootstrap.cc:492] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.353343  4521 tablet_bootstrap.cc:492] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.353479  4523 consensus_queue.cc:260] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.359549  4523 ts_tablet_manager.cc:1434] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 12:23:05.360641  4523 tablet_bootstrap.cc:492] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.361017  4521 tablet_bootstrap.cc:654] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.361763  4522 tablet_bootstrap.cc:654] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.369150  4523 tablet_bootstrap.cc:654] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.374825  4522 tablet_bootstrap.cc:492] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.375562  4522 ts_tablet_manager.cc:1403] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 12:23:05.376682  4521 tablet_bootstrap.cc:492] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.377286  4521 ts_tablet_manager.cc:1403] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.013s
I20260709 12:23:05.379163  4522 raft_consensus.cc:359] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.380084  4522 raft_consensus.cc:385] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.380440  4522 raft_consensus.cc:740] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.380383  4521 raft_consensus.cc:359] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.381554  4521 raft_consensus.cc:385] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.381927  4521 raft_consensus.cc:740] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.381935  4522 consensus_queue.cc:260] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.383024  4521 consensus_queue.cc:260] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.387151  4522 ts_tablet_manager.cc:1434] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:23:05.388510  4522 tablet_bootstrap.cc:492] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.385577  4521 ts_tablet_manager.cc:1434] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:05.392139  4521 tablet_bootstrap.cc:492] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.395674  4522 tablet_bootstrap.cc:654] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.399294  4521 tablet_bootstrap.cc:654] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.403177  4507 raft_consensus.cc:493] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.403996  4507 raft_consensus.cc:515] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.408488  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:05.409478  4223 raft_consensus.cc:2468] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:05.411494  4521 tablet_bootstrap.cc:492] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.411728  4264 leader_election.cc:304] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.412469  4521 ts_tablet_manager.cc:1403] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 12:23:05.415823  4530 raft_consensus.cc:493] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.416239  4521 raft_consensus.cc:359] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.416517  4530 raft_consensus.cc:515] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.422601  4530 leader_election.cc:290] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:05.417105  4521 raft_consensus.cc:385] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.424476  4521 raft_consensus.cc:740] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.425292  4507 leader_election.cc:290] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.425554  4521 consensus_queue.cc:260] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.427147  4523 tablet_bootstrap.cc:492] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.427718  4523 ts_tablet_manager.cc:1403] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.004s
I20260709 12:23:05.427791  4546 raft_consensus.cc:2804] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.428498  4546 raft_consensus.cc:493] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.428845  4546 raft_consensus.cc:3060] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.431850  4523 raft_consensus.cc:359] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.432785  4523 raft_consensus.cc:385] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.433113  4523 raft_consensus.cc:740] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.435611  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.433982  4523 consensus_queue.cc:260] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.436586  4371 raft_consensus.cc:2468] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:05.423650  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:05.438380  4223 raft_consensus.cc:2468] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:05.438807  4523 ts_tablet_manager.cc:1434] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:23:05.439234  4521 ts_tablet_manager.cc:1434] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:23:05.439697  4338 leader_election.cc:304] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:05.440284  4523 tablet_bootstrap.cc:492] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.442353  4530 raft_consensus.cc:2804] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.442898  4530 raft_consensus.cc:493] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.443264  4530 raft_consensus.cc:3060] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.448822  4523 tablet_bootstrap.cc:654] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.451972  4546 raft_consensus.cc:515] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.458595  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:05.459429  4298 raft_consensus.cc:2468] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:05.460660  4521 tablet_bootstrap.cc:492] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.464768  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:05.466243  4223 raft_consensus.cc:3060] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.468113  4523 tablet_bootstrap.cc:492] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.468725  4523 ts_tablet_manager.cc:1403] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.008s
I20260709 12:23:05.469738  4530 raft_consensus.cc:515] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.464768  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:05.470785  4371 raft_consensus.cc:3060] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.471939  4523 raft_consensus.cc:359] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.472829  4523 raft_consensus.cc:385] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.473204  4523 raft_consensus.cc:740] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.473161  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:05.474037  4222 raft_consensus.cc:3060] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.474284  4523 consensus_queue.cc:260] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.464867  4546 leader_election.cc:290] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.477177  4523 ts_tablet_manager.cc:1434] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:05.478032  4522 tablet_bootstrap.cc:492] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.478536  4523 tablet_bootstrap.cc:492] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.478720  4522 ts_tablet_manager.cc:1403] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.090s	user 0.030s	sys 0.006s
I20260709 12:23:05.482332  4522 raft_consensus.cc:359] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.483031  4458 raft_consensus.cc:493] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.483131  4522 raft_consensus.cc:385] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.483809  4522 raft_consensus.cc:740] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.483696  4458 raft_consensus.cc:515] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.484972  4522 consensus_queue.cc:260] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.486348  4523 tablet_bootstrap.cc:654] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.487867  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.488355  4521 tablet_bootstrap.cc:654] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.486300  4458 leader_election.cc:290] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.488615  4369 raft_consensus.cc:2468] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:05.490221  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:05.490909  4298 raft_consensus.cc:2468] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:05.492087  4186 leader_election.cc:304] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:05.490861  4222 raft_consensus.cc:2468] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:05.495030  4338 leader_election.cc:304] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:05.495164  4530 leader_election.cc:290] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:05.495108  4458 raft_consensus.cc:2804] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.495975  4458 raft_consensus.cc:493] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.497159  4458 raft_consensus.cc:3060] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.495998  4530 raft_consensus.cc:2804] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.498224  4523 tablet_bootstrap.cc:492] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.498761  4523 ts_tablet_manager.cc:1403] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.011s
I20260709 12:23:05.498816  4530 raft_consensus.cc:697] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:05.501670  4523 raft_consensus.cc:359] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.502607  4523 raft_consensus.cc:385] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.502928  4523 raft_consensus.cc:740] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.503894  4523 consensus_queue.cc:260] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.499843  4530 consensus_queue.cc:237] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.503989  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:05.505513  4298 raft_consensus.cc:3060] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.512145  4521 tablet_bootstrap.cc:492] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.512892  4522 ts_tablet_manager.cc:1434] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 12:23:05.513275  4523 ts_tablet_manager.cc:1434] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.008s
I20260709 12:23:05.512890  4521 ts_tablet_manager.cc:1403] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.004s
I20260709 12:23:05.514703  4522 tablet_bootstrap.cc:492] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.514832  4523 tablet_bootstrap.cc:492] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d: Bootstrap starting.
I20260709 12:23:05.519438  4458 raft_consensus.cc:515] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.522789  4523 tablet_bootstrap.cc:654] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.523025  4297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:05.523780  4297 raft_consensus.cc:3060] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.525645  4522 tablet_bootstrap.cc:654] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.528323  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:05.529029  4369 raft_consensus.cc:3060] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.493489  4223 raft_consensus.cc:2468] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:05.538690  4369 raft_consensus.cc:2468] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:05.540302  4264 leader_election.cc:304] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.540233  4521 raft_consensus.cc:359] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.542387  4186 leader_election.cc:304] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:05.543478  4552 raft_consensus.cc:2804] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.543632  4546 raft_consensus.cc:2804] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.543514  4371 raft_consensus.cc:2468] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:05.543548  4297 raft_consensus.cc:2468] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:05.545367  4546 raft_consensus.cc:697] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:05.539333  4458 leader_election.cc:290] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.546293  4546 consensus_queue.cc:237] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.543835  4298 raft_consensus.cc:2468] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
W20260709 12:23:05.565811  4169 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:05.542069  4521 raft_consensus.cc:385] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.572369  4552 raft_consensus.cc:697] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:05.572444  4521 raft_consensus.cc:740] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.573628  4552 consensus_queue.cc:237] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.573800  4521 consensus_queue.cc:260] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.571017  4117 catalog_manager.cc:5697] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:05.577906  4523 tablet_bootstrap.cc:492] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d: No bootstrap required, opened a new log
I20260709 12:23:05.580482  4523 ts_tablet_manager.cc:1403] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.028s
I20260709 12:23:05.583693  4522 tablet_bootstrap.cc:492] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.579073  4521 ts_tablet_manager.cc:1434] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.002s
I20260709 12:23:05.584393  4522 ts_tablet_manager.cc:1403] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.070s	user 0.024s	sys 0.020s
I20260709 12:23:05.585253  4118 catalog_manager.cc:5697] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.586313  4523 raft_consensus.cc:359] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.587949  4522 raft_consensus.cc:359] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.588336  4523 raft_consensus.cc:385] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.588670  4522 raft_consensus.cc:385] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.591244  4522 raft_consensus.cc:740] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.592109  4522 consensus_queue.cc:260] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.594411  4522 ts_tablet_manager.cc:1434] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.001s
I20260709 12:23:05.594627  4523 raft_consensus.cc:740] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.595624  4522 tablet_bootstrap.cc:492] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17: Bootstrap starting.
I20260709 12:23:05.595582  4523 consensus_queue.cc:260] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.598021  4523 ts_tablet_manager.cc:1434] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:23:05.585320  4521 tablet_bootstrap.cc:492] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.604988  4522 tablet_bootstrap.cc:654] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.602962  4116 catalog_manager.cc:5697] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 0 to 1, leader changed from <none> to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 1 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.610468  4521 tablet_bootstrap.cc:654] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.612814  4547 raft_consensus.cc:493] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.613417  4547 raft_consensus.cc:515] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.617205  4521 tablet_bootstrap.cc:492] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.618000  4521 ts_tablet_manager.cc:1403] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.005s
I20260709 12:23:05.620306  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.621165  4371 raft_consensus.cc:2468] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:05.622601  4547 leader_election.cc:290] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.621766  4521 raft_consensus.cc:359] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.624560  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:05.625149  4552 raft_consensus.cc:493] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.625800  4546 raft_consensus.cc:493] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.629045  4522 tablet_bootstrap.cc:492] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17: No bootstrap required, opened a new log
I20260709 12:23:05.628651  4552 raft_consensus.cc:515] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.628671  4546 raft_consensus.cc:515] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.627943  4521 raft_consensus.cc:385] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.629591  4522 ts_tablet_manager.cc:1403] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.019s
I20260709 12:23:05.629772  4521 raft_consensus.cc:740] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.630818  4521 consensus_queue.cc:260] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.631150  4552 leader_election.cc:290] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.632499  4522 raft_consensus.cc:359] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.633148  4521 ts_tablet_manager.cc:1434] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:23:05.633262  4522 raft_consensus.cc:385] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.633662  4223 raft_consensus.cc:2468] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:05.635107  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.635820  4371 raft_consensus.cc:2468] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:05.636546  4522 raft_consensus.cc:740] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.634389  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40760ebaa0e422c881e8efcbcbe83d4" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:05.631206  4546 leader_election.cc:290] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.636200  4521 tablet_bootstrap.cc:492] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274: Bootstrap starting.
I20260709 12:23:05.625908  4261 leader_election.cc:304] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.637660  4186 leader_election.cc:304] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:05.638365  4222 raft_consensus.cc:2468] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:05.639365  4458 raft_consensus.cc:2804] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.639811  4458 raft_consensus.cc:493] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.640187  4458 raft_consensus.cc:3060] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.643026  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:05.643698  4555 raft_consensus.cc:493] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.643764  4298 raft_consensus.cc:2468] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:05.644383  4555 raft_consensus.cc:515] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.646937  4264 leader_election.cc:304] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.647687  4546 raft_consensus.cc:2804] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.648808  4507 raft_consensus.cc:2804] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.649956  4546 raft_consensus.cc:493] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.650143  4507 raft_consensus.cc:493] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.650776  4507 raft_consensus.cc:3060] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.651577  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.652289  4369 raft_consensus.cc:2468] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:05.653579  4186 leader_election.cc:304] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:05.653959  4546 raft_consensus.cc:3060] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.654105  4521 tablet_bootstrap.cc:654] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.654779  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40760ebaa0e422c881e8efcbcbe83d4" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.655561  4371 raft_consensus.cc:2468] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:05.649413  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:05.637662  4522 consensus_queue.cc:260] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.650228  4555 leader_election.cc:290] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.659097  4522 ts_tablet_manager.cc:1434] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 12:23:05.657573  4298 raft_consensus.cc:2468] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 0.
I20260709 12:23:05.666283  4546 raft_consensus.cc:515] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.670656  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:05.664697  4458 raft_consensus.cc:515] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.665107  4552 raft_consensus.cc:2804] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.672946  4552 raft_consensus.cc:493] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.672662  4507 raft_consensus.cc:515] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.673425  4552 raft_consensus.cc:3060] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.674230  4222 raft_consensus.cc:2393] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 53ebf9844ebb4152846b6e9fd1249a17 in current term 1: Already voted for candidate 50159f6246ed4d19bc4169a9c142b98d in this term.
I20260709 12:23:05.676827  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:05.677434  4371 raft_consensus.cc:3060] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.673547  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:05.681195  4264 leader_election.cc:304] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:05.681749  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d385031a46bf491cabbcc60263ac6e9a" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:05.682615  4547 raft_consensus.cc:2749] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.683424  4548 raft_consensus.cc:493] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.684078  4298 raft_consensus.cc:2393] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 50159f6246ed4d19bc4169a9c142b98d in current term 1: Already voted for candidate 53ebf9844ebb4152846b6e9fd1249a17 in this term.
I20260709 12:23:05.684154  4548 raft_consensus.cc:515] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.687779  4548 leader_election.cc:290] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.688732  4555 raft_consensus.cc:493] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.689360  4555 raft_consensus.cc:515] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.680835  4530 consensus_queue.cc:1048] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:23:05.692298  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.692452  4458 leader_election.cc:290] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
W20260709 12:23:05.694746  4261 leader_election.cc:343] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.695753  4371 raft_consensus.cc:2468] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:05.696841  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:05.697863  4222 raft_consensus.cc:2468] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 0.
I20260709 12:23:05.698868  4186 leader_election.cc:304] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:05.699579  4264 leader_election.cc:304] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:05.701385  4548 raft_consensus.cc:2804] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.701996  4548 raft_consensus.cc:493] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.702107  4530 raft_consensus.cc:493] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.702577  4458 raft_consensus.cc:2804] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.702586  4548 raft_consensus.cc:3060] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.703128  4458 raft_consensus.cc:697] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:05.702832  4530 raft_consensus.cc:515] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.704535  4458 consensus_queue.cc:237] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.707968  4530 leader_election.cc:290] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:05.710095  4297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:05.711045  4297 raft_consensus.cc:2468] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:05.713326  4338 leader_election.cc:304] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.714066  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40760ebaa0e422c881e8efcbcbe83d4" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:05.717006  4369 raft_consensus.cc:3060] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.721032  4297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17" is_pre_election: true
I20260709 12:23:05.715773  4530 raft_consensus.cc:2804] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.715939  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40760ebaa0e422c881e8efcbcbe83d4" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:05.721784  4530 raft_consensus.cc:493] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.722174  4223 raft_consensus.cc:3060] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.722442  4530 raft_consensus.cc:3060] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.722831  4507 leader_election.cc:290] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.714915  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d" is_pre_election: true
I20260709 12:23:05.725710  4222 raft_consensus.cc:2468] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 0.
I20260709 12:23:05.718236  4555 leader_election.cc:290] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.720094  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274" is_pre_election: true
I20260709 12:23:05.740727  4546 leader_election.cc:290] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
W20260709 12:23:05.742010  4186 leader_election.cc:343] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.748167  4521 tablet_bootstrap.cc:492] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274: No bootstrap required, opened a new log
I20260709 12:23:05.748769  4521 ts_tablet_manager.cc:1403] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent bootstrapping tablet: real 0.113s	user 0.047s	sys 0.025s
I20260709 12:23:05.751780  4521 raft_consensus.cc:359] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.752532  4521 raft_consensus.cc:385] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.751852  4530 raft_consensus.cc:515] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.758234  4552 raft_consensus.cc:515] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.752664  4223 raft_consensus.cc:2468] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:05.752874  4521 raft_consensus.cc:740] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.761565  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:05.762833  4298 raft_consensus.cc:3060] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.763535  4548 raft_consensus.cc:515] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.764830  4297 raft_consensus.cc:2393] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50159f6246ed4d19bc4169a9c142b98d in current term 1: Already voted for candidate 53ebf9844ebb4152846b6e9fd1249a17 in this term.
I20260709 12:23:05.767508  4189 leader_election.cc:304] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d; no voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:05.768419  4555 raft_consensus.cc:3060] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.770912  4264 leader_election.cc:304] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.772531  4521 consensus_queue.cc:260] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.774704  4546 raft_consensus.cc:2804] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.776168  4546 raft_consensus.cc:697] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:05.776400  4369 raft_consensus.cc:2468] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:05.777263  4546 consensus_queue.cc:237] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.774897  4521 ts_tablet_manager.cc:1434] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:23:05.771777  4298 raft_consensus.cc:2468] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:05.781916  4189 leader_election.cc:304] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.782843  4548 leader_election.cc:290] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.783022  4561 raft_consensus.cc:2804] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.784610  4552 leader_election.cc:290] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 1 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:05.785144  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:05.785892  4371 raft_consensus.cc:3060] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.789356  4369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:05.789597  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:05.791416  4369 raft_consensus.cc:3060] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.795228  4371 raft_consensus.cc:2468] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 1.
I20260709 12:23:05.780637  4555 raft_consensus.cc:2749] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.799481  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:05.800102  4222 raft_consensus.cc:3060] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.801190  4369 raft_consensus.cc:2468] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:05.802032  4530 leader_election.cc:290] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:05.803185  4261 leader_election.cc:304] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.803846  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:05.804503  4298 raft_consensus.cc:3060] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.811826  4507 raft_consensus.cc:2804] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.812458  4507 raft_consensus.cc:697] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:05.812563  4298 raft_consensus.cc:2468] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:05.813438  4507 consensus_queue.cc:237] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.813973  4338 leader_election.cc:304] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:05.814927  4530 raft_consensus.cc:2804] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.815622  4530 raft_consensus.cc:697] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:05.815403  4222 raft_consensus.cc:2468] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 1.
I20260709 12:23:05.816628  4530 consensus_queue.cc:237] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.809423  4116 catalog_manager.cc:5697] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 0 to 1, leader changed from <none> to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 1 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.833910  4223 raft_consensus.cc:2468] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 1.
I20260709 12:23:05.849054  4561 raft_consensus.cc:697] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:05.865056  4118 catalog_manager.cc:5697] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.868999  4561 consensus_queue.cc:237] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:05.887647  4118 catalog_manager.cc:5697] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 1 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.889511  4116 catalog_manager.cc:5697] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 1 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:05.896518  4536 consensus_queue.cc:1048] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.899247  4117 catalog_manager.cc:5697] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 0 to 1, leader changed from <none> to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 1 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.960933  4561 consensus_queue.cc:1048] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.973582  4567 consensus_queue.cc:1048] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.991035  4536 consensus_queue.cc:1048] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.003378  4571 consensus_queue.cc:1048] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:06.016482  4507 consensus_queue.cc:1048] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.017606  4546 consensus_queue.cc:1048] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.089931  4369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
mode: GRACEFUL
new_leader_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:54400
I20260709 12:23:06.090701  4369 raft_consensus.cc:606] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Received request to transfer leadership to TS 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:06.097620  4369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
mode: GRACEFUL
new_leader_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:54400
I20260709 12:23:06.098204  4369 raft_consensus.cc:606] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Received request to transfer leadership to TS 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:06.107625  4297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
mode: GRACEFUL
new_leader_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:41048
I20260709 12:23:06.108211  4297 raft_consensus.cc:606] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Received request to transfer leadership to TS 50159f6246ed4d19bc4169a9c142b98d
I20260709 12:23:06.114108  4222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
mode: GRACEFUL
new_leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:35626
I20260709 12:23:06.114816  4222 raft_consensus.cc:606] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Received request to transfer leadership to TS 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:06.120689  4222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
mode: GRACEFUL
new_leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:35626
I20260709 12:23:06.121236  4222 raft_consensus.cc:606] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Received request to transfer leadership to TS 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:06.130250  4369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
mode: GRACEFUL
new_leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:54400
I20260709 12:23:06.131060  4369 raft_consensus.cc:606] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Received request to transfer leadership to TS 53ebf9844ebb4152846b6e9fd1249a17
I20260709 12:23:06.132462  4458 consensus_queue.cc:1048] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.137248  4298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
mode: GRACEFUL
new_leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:41048
I20260709 12:23:06.137982  4298 raft_consensus.cc:606] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Received request to transfer leadership to TS 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:06.145820  4458 consensus_queue.cc:1048] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.263229  4571 consensus_queue.cc:1048] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.271189  4530 raft_consensus.cc:993] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: : Instructing follower 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:06.271857  4536 raft_consensus.cc:1081] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Signalling peer 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:06.274524  4298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:41046
I20260709 12:23:06.277874  4298 raft_consensus.cc:493] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:06.278291  4298 raft_consensus.cc:3060] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.280536  4536 consensus_queue.cc:1048] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:06.287230  4298 raft_consensus.cc:515] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.294253  4298 leader_election.cc:290] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:06.294173  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:06.295759  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "752c55dfc1984731b3f2604f63ccae71" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:06.296424  4222 raft_consensus.cc:3060] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.296465  4372 raft_consensus.cc:3055] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:06.297159  4372 raft_consensus.cc:740] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:06.298101  4372 consensus_queue.cc:260] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.299489  4372 raft_consensus.cc:3060] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.304991  4222 raft_consensus.cc:2468] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:06.306632  4547 consensus_queue.cc:1048] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:06.309067  4264 leader_election.cc:304] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:06.309157  4372 raft_consensus.cc:2468] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:06.310236  4547 raft_consensus.cc:2804] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:06.312067  4547 raft_consensus.cc:697] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:06.313659  4547 consensus_queue.cc:237] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.320564  4547 consensus_queue.cc:1048] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.324594  4117 catalog_manager.cc:5697] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 1 to 2, leader changed from 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67) to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.341912  4547 consensus_queue.cc:1048] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:06.354518  4546 raft_consensus.cc:993] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: : Instructing follower 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:06.355191  4548 raft_consensus.cc:1081] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Signalling peer 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:06.357995  4372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:54376
I20260709 12:23:06.358861  4372 raft_consensus.cc:493] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:06.359247  4372 raft_consensus.cc:3060] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.360602  4548 consensus_queue.cc:1048] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.365450  4552 consensus_queue.cc:1048] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.379745  4458 consensus_queue.cc:1048] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.380012  4372 raft_consensus.cc:515] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.381333  4371 raft_consensus.cc:1240] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Rejecting Update request from peer 53ebf9844ebb4152846b6e9fd1249a17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:06.383611  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:06.384400  4222 raft_consensus.cc:3060] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.385798  4298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3745b69c0304470b91ff96889e8485f2" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:06.386555  4298 raft_consensus.cc:3055] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:06.386919  4298 raft_consensus.cc:740] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:06.387766  4298 consensus_queue.cc:260] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.388995  4298 raft_consensus.cc:3060] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.391225  4536 raft_consensus.cc:993] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: : Instructing follower 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:06.391786  4536 raft_consensus.cc:1081] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Signalling peer 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:06.391781  4222 raft_consensus.cc:2468] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 2.
I20260709 12:23:06.393682  4338 leader_election.cc:304] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:06.394666  4530 raft_consensus.cc:2804] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:06.394129  4222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:35624
I20260709 12:23:06.395219  4530 raft_consensus.cc:697] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:06.395352  4458 raft_consensus.cc:993] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: : Instructing follower 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:06.395944  4458 raft_consensus.cc:1081] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Signalling peer 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:06.396194  4530 consensus_queue.cc:237] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.399124  4297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:41034
I20260709 12:23:06.399757  4297 raft_consensus.cc:493] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:06.395352  4222 raft_consensus.cc:493] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:06.400537  4222 raft_consensus.cc:3060] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.400172  4297 raft_consensus.cc:3060] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.410193  4222 raft_consensus.cc:515] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.409693  4117 catalog_manager.cc:5697] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 1 to 2, leader changed from 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66) to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 2 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:06.413897  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:06.414465  4371 raft_consensus.cc:3055] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:06.414780  4371 raft_consensus.cc:740] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:06.414954  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a960b1c7238842a5b31e0f5d208287cb" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:06.415536  4371 consensus_queue.cc:260] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.416831  4371 raft_consensus.cc:3060] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.417789  4298 raft_consensus.cc:2468] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 2.
I20260709 12:23:06.419747  4372 leader_election.cc:290] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:06.425078  4371 raft_consensus.cc:2468] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:06.426018  4222 leader_election.cc:290] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:06.426707  4186 leader_election.cc:304] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:06.415563  4296 raft_consensus.cc:3060] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.427841  4458 raft_consensus.cc:2804] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:06.428419  4458 raft_consensus.cc:697] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:06.428712  4530 raft_consensus.cc:993] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: : Instructing follower 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:06.429293  4571 raft_consensus.cc:1081] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Signalling peer 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:06.429345  4458 consensus_queue.cc:237] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.431339  4223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:35624
I20260709 12:23:06.431927  4223 raft_consensus.cc:493] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:06.432250  4223 raft_consensus.cc:3060] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.435268  4296 raft_consensus.cc:2468] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:06.439666  4297 raft_consensus.cc:515] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.440784  4223 raft_consensus.cc:515] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.442912  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:06.443403  4297 leader_election.cc:290] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:06.446360  4223 leader_election.cc:290] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:06.446730  4372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:06.447474  4372 raft_consensus.cc:3055] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:06.447841  4372 raft_consensus.cc:740] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:06.442934  4116 catalog_manager.cc:5697] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 1 to 2, leader changed from 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67) to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 2 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.448731  4372 consensus_queue.cc:260] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.449864  4372 raft_consensus.cc:3060] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.445286  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2263f966bfcd4876ad9943d998fab653" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:06.445453  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb163fdfe074544b7eb876f26ec6570" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:06.451440  4296 raft_consensus.cc:3060] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.443791  4222 raft_consensus.cc:3055] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:06.454756  4222 raft_consensus.cc:740] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:06.455653  4222 consensus_queue.cc:260] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.456092  4372 raft_consensus.cc:2468] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:06.457098  4222 raft_consensus.cc:3060] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.457314  4186 leader_election.cc:304] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:06.457451  4371 raft_consensus.cc:3060] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.459318  4458 raft_consensus.cc:2804] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:06.460042  4458 raft_consensus.cc:697] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:06.460924  4458 consensus_queue.cc:237] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.464617  4222 raft_consensus.cc:2468] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:06.465382  4371 raft_consensus.cc:2468] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:06.466005  4296 raft_consensus.cc:2468] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:06.466171  4264 leader_election.cc:304] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50159f6246ed4d19bc4169a9c142b98d, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:06.467162  4548 raft_consensus.cc:2804] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:06.467666  4548 raft_consensus.cc:697] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:06.468447  4548 consensus_queue.cc:237] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.472910  4116 catalog_manager.cc:5697] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 1 to 2, leader changed from 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67) to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 2 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.476104  4117 catalog_manager.cc:5697] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 1 to 2, leader changed from 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65) to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.485219  4546 raft_consensus.cc:993] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: : Instructing follower 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:06.485695  4548 raft_consensus.cc:1081] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Signalling peer 50159f6246ed4d19bc4169a9c142b98d to start an election
I20260709 12:23:06.487460  4222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
 from {username='slave'} at 127.0.0.1:35618
I20260709 12:23:06.487973  4222 raft_consensus.cc:493] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:06.488313  4222 raft_consensus.cc:3060] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.494479  4222 raft_consensus.cc:515] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.496501  4222 leader_election.cc:290] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Requested vote from peers 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:06.497471  4296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:06.497829  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ecb652aac54a2fbb426cd63803e74a" candidate_uuid: "50159f6246ed4d19bc4169a9c142b98d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:06.498234  4296 raft_consensus.cc:3055] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:06.498553  4371 raft_consensus.cc:3060] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.498898  4296 raft_consensus.cc:740] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:06.499831  4296 consensus_queue.cc:260] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.501158  4296 raft_consensus.cc:3060] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.506898  4371 raft_consensus.cc:2468] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:06.508208  4186 leader_election.cc:304] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:06.508922  4296 raft_consensus.cc:2468] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50159f6246ed4d19bc4169a9c142b98d in term 2.
I20260709 12:23:06.509150  4458 raft_consensus.cc:2804] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:06.509822  4458 raft_consensus.cc:697] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Becoming Leader. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:06.510749  4458 consensus_queue.cc:237] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.517724  4117 catalog_manager.cc:5697] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d reported cstate change: term changed from 1 to 2, leader changed from 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66) to 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65). New cstate: current_term: 2 leader_uuid: "50159f6246ed4d19bc4169a9c142b98d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.571048  4555 raft_consensus.cc:993] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: : Instructing follower 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:06.571784  4567 raft_consensus.cc:1081] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Signalling peer 53ebf9844ebb4152846b6e9fd1249a17 to start an election
I20260709 12:23:06.573796  4296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
 from {username='slave'} at 127.0.0.1:41034
I20260709 12:23:06.574453  4296 raft_consensus.cc:493] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:06.574879  4296 raft_consensus.cc:3060] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.583904  4296 raft_consensus.cc:515] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.586833  4296 leader_election.cc:290] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67:33495)
I20260709 12:23:06.587412  4371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
I20260709 12:23:06.587255  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68d9f3528b479baf8a639760570f70" candidate_uuid: "53ebf9844ebb4152846b6e9fd1249a17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:06.588743  4371 raft_consensus.cc:3060] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.588837  4222 raft_consensus.cc:3055] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:06.589273  4222 raft_consensus.cc:740] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:06.590154  4222 consensus_queue.cc:260] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.591545  4222 raft_consensus.cc:3060] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:06.598348  4371 raft_consensus.cc:2468] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:06.599654  4222 raft_consensus.cc:2468] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ebf9844ebb4152846b6e9fd1249a17 in term 2.
I20260709 12:23:06.599874  4261 leader_election.cc:304] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:06.601052  4548 raft_consensus.cc:2804] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:06.601554  4548 raft_consensus.cc:697] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Leader. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:06.602425  4548 consensus_queue.cc:237] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:06.610343  4117 catalog_manager.cc:5697] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 reported cstate change: term changed from 1 to 2, leader changed from 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65) to 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66). New cstate: current_term: 2 leader_uuid: "53ebf9844ebb4152846b6e9fd1249a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.699376  4222 raft_consensus.cc:1275] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.700990  4548 consensus_queue.cc:1048] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.709419  4372 raft_consensus.cc:1275] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.710902  4547 consensus_queue.cc:1048] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.884850  4296 raft_consensus.cc:1275] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.886348  4536 consensus_queue.cc:1048] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.897317  4222 raft_consensus.cc:1275] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.898819  4536 consensus_queue.cc:1048] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.907135  4372 raft_consensus.cc:1275] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.908506  4567 consensus_queue.cc:1048] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.916816  4296 raft_consensus.cc:1275] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.919001  4567 consensus_queue.cc:1048] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.924818  4297 raft_consensus.cc:1275] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.926016  4567 consensus_queue.cc:1048] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.937881  4297 raft_consensus.cc:1275] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.939625  4592 consensus_queue.cc:1048] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.942729  4372 raft_consensus.cc:1275] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.944187  4592 consensus_queue.cc:1048] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:06.954910  4371 raft_consensus.cc:1275] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 50159f6246ed4d19bc4169a9c142b98d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:06.956323  4458 consensus_queue.cc:1048] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:07.005173  4222 raft_consensus.cc:1275] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:07.007021  4547 consensus_queue.cc:1048] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:07.015475  4371 raft_consensus.cc:1275] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:07.017318  4548 consensus_queue.cc:1048] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:07.031857  4371 raft_consensus.cc:1275] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:07.033203  4548 consensus_queue.cc:1048] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:07.041909  4222 raft_consensus.cc:1275] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Refusing update from remote peer 53ebf9844ebb4152846b6e9fd1249a17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:07.043812  4547 consensus_queue.cc:1048] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:09.141917  4117 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40272:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:23:09.165454  4117 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40272:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:23:09.189692 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:09.430024  4628 log.cc:927] Time spent T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: Append to log took a long time: real 0.143s	user 0.005s	sys 0.000s
W20260709 12:23:09.431865  4635 log.cc:927] Time spent T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:09.559374  4657 tablet.cc:1724] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.554839  4659 tablet.cc:1724] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:23:09.598155  4648 log.cc:927] Time spent T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
W20260709 12:23:09.629522  4643 log.cc:927] Time spent T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
W20260709 12:23:09.623905  4264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.65:36539, user_credentials={real_user=slave}} blocked reactor thread for 67743.9us
W20260709 12:23:09.614578  4261 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.23.217.67:33495, user_credentials={real_user=slave}} blocked reactor thread for 62465.2us
I20260709 12:23:09.618146  4656 tablet.cc:1724] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.665115  4665 tablet.cc:1724] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.661293  4660 tablet.cc:1724] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.654691  4663 tablet.cc:1724] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.708997  4670 tablet.cc:1724] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.700016  4664 tablet.cc:1724] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.720276  4298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21"
dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
mode: GRACEFUL
new_leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:51228
I20260709 12:23:09.724906  4667 tablet.cc:1724] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.727322  4298 raft_consensus.cc:606] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Received request to transfer leadership to TS 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:09.703510  4666 tablet.cc:1724] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.739710  4671 tablet.cc:1724] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.755363  4636 raft_consensus.cc:993] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17: : Instructing follower 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:09.756016  4636 raft_consensus.cc:1081] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Signalling peer 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:09.761041  4365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:54376
I20260709 12:23:09.761726  4365 raft_consensus.cc:493] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:09.762123  4365 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:09.753100  4659 tablet.cc:1724] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.766798  4656 tablet.cc:1724] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.786451  4666 tablet.cc:1724] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.793059  4670 tablet.cc:1724] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.793226  4671 tablet.cc:1724] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.794634  4669 tablet.cc:1724] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.800043  4660 tablet.cc:1724] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.806727  4666 tablet.cc:1724] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.813954  4663 tablet.cc:1724] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.817416  4665 tablet.cc:1724] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.818053  4671 tablet.cc:1724] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.820266  4664 tablet.cc:1724] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.821961  4669 tablet.cc:1724] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.772295  4657 tablet.cc:1724] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.830685  4365 raft_consensus.cc:515] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:09.838523  4659 tablet.cc:1724] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.829470  4667 tablet.cc:1724] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:23:09.846427  4365 leader_election.cc:290] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 3 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:09.848650  4218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:09.859216  4218 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:09.847697  4295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722efbbd5c784722a3277e0fb9453f21" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:09.870355  4295 raft_consensus.cc:3055] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:09.870841  4295 raft_consensus.cc:740] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53ebf9844ebb4152846b6e9fd1249a17, State: Running, Role: LEADER
I20260709 12:23:09.872010  4295 consensus_queue.cc:260] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:09.879623  4295 raft_consensus.cc:3060] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:09.884862  4218 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 3.
I20260709 12:23:09.892326  4338 leader_election.cc:304] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 50159f6246ed4d19bc4169a9c142b98d; no voters: 
I20260709 12:23:09.894132  4600 raft_consensus.cc:2804] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:09.894810  4600 raft_consensus.cc:697] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:09.895695  4600 consensus_queue.cc:237] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:09.908118  4218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f936823aef084ec084d006f0af36a144"
dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
mode: GRACEFUL
new_leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:48812
I20260709 12:23:09.909017  4218 raft_consensus.cc:606] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Received request to transfer leadership to TS 3e5c80c10c344270b0d22ed21ccb1274
I20260709 12:23:09.910668 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:09.911584 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:09.905277  4295 raft_consensus.cc:2468] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 3.
I20260709 12:23:09.921309  4118 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a: soft_deleted_table [id=319e2f5eef344f0db315a975ee3460d2] alter complete (version 1)
I20260709 12:23:09.930996  4118 catalog_manager.cc:5697] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 2 to 3, leader changed from 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66) to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:09.958786  4599 raft_consensus.cc:993] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d: : Instructing follower 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:09.959393  4638 raft_consensus.cc:1081] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Signalling peer 3e5c80c10c344270b0d22ed21ccb1274 to start an election
I20260709 12:23:09.961086  4365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f936823aef084ec084d006f0af36a144"
dest_uuid: "3e5c80c10c344270b0d22ed21ccb1274"
 from {username='slave'} at 127.0.0.1:54388
I20260709 12:23:09.961746  4365 raft_consensus.cc:493] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:09.962193  4365 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:09.968326  4365 raft_consensus.cc:515] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:09.970000  4365 leader_election.cc:290] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 3 election: Requested vote from peers 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539), 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909)
I20260709 12:23:09.970841  4218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50159f6246ed4d19bc4169a9c142b98d"
I20260709 12:23:09.971251  4295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f936823aef084ec084d006f0af36a144" candidate_uuid: "3e5c80c10c344270b0d22ed21ccb1274" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53ebf9844ebb4152846b6e9fd1249a17"
I20260709 12:23:09.971661  4218 raft_consensus.cc:3055] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:09.972132  4295 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:09.972187  4218 raft_consensus.cc:740] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50159f6246ed4d19bc4169a9c142b98d, State: Running, Role: LEADER
I20260709 12:23:09.973212  4218 consensus_queue.cc:260] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:09.974280  4218 raft_consensus.cc:3060] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:09.978287  4295 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 3.
I20260709 12:23:09.979643  4218 raft_consensus.cc:2468] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e5c80c10c344270b0d22ed21ccb1274 in term 3.
I20260709 12:23:09.979710  4338 leader_election.cc:304] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e5c80c10c344270b0d22ed21ccb1274, 53ebf9844ebb4152846b6e9fd1249a17; no voters: 
I20260709 12:23:09.980785  4600 raft_consensus.cc:2804] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:09.981309  4600 raft_consensus.cc:697] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 LEADER]: Becoming Leader. State: Replica: 3e5c80c10c344270b0d22ed21ccb1274, State: Running, Role: LEADER
I20260709 12:23:09.982084  4600 consensus_queue.cc:237] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } }
I20260709 12:23:09.990756  4117 catalog_manager.cc:5697] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 reported cstate change: term changed from 2 to 3, leader changed from 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65) to 3e5c80c10c344270b0d22ed21ccb1274 (127.23.217.67). New cstate: current_term: 3 leader_uuid: "3e5c80c10c344270b0d22ed21ccb1274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e5c80c10c344270b0d22ed21ccb1274" member_type: VOTER last_known_addr { host: "127.23.217.67" port: 33495 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:10.384397  4223 raft_consensus.cc:1275] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:10.385886  4600 consensus_queue.cc:1048] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:10.388305  4295 raft_consensus.cc:1275] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:10.389921  4600 consensus_queue.cc:1048] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:10.415901  4298 raft_consensus.cc:1275] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:10.426211  4571 consensus_queue.cc:1048] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ebf9844ebb4152846b6e9fd1249a17" member_type: VOTER last_known_addr { host: "127.23.217.66" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:10.434952  4223 raft_consensus.cc:1275] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Refusing update from remote peer 3e5c80c10c344270b0d22ed21ccb1274: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:10.436544  4600 consensus_queue.cc:1048] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50159f6246ed4d19bc4169a9c142b98d" member_type: VOTER last_known_addr { host: "127.23.217.65" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:10.912951 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:10.918649 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:10.919159 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:11.920387 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:11.927208 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:11.928031 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:12.929448 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:12.935597 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:12.936079 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:13.937268 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:13.943766 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:13.944329 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:14.945758 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:14.952677 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:14.953132 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:15.954263 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:15.959429 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:15.959811 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:16.960721 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:16.967530 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:16.968060 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:17.969077 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:17.974159 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:17.974598 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:18.975584 24421 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:18.980971 24421 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:18.981506 24421 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:20.043371 24421 tablet_server.cc:179] TabletServer@127.23.217.65:0 shutting down...
I20260709 12:23:20.093580 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:20.094604 24421 tablet_replica.cc:333] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.095728 24421 raft_consensus.cc:2243] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.096421 24421 raft_consensus.cc:2272] T 752c55dfc1984731b3f2604f63ccae71 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.100682 24421 tablet_replica.cc:333] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.101410 24421 raft_consensus.cc:2243] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.102065 24421 raft_consensus.cc:2272] T c40760ebaa0e422c881e8efcbcbe83d4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.104765 24421 tablet_replica.cc:333] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.105355 24421 raft_consensus.cc:2243] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.106343 24421 raft_consensus.cc:2272] T 97ecb652aac54a2fbb426cd63803e74a P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.109084 24421 tablet_replica.cc:333] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.109746 24421 raft_consensus.cc:2243] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.110296 24421 raft_consensus.cc:2272] T 2263f966bfcd4876ad9943d998fab653 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.112560 24421 tablet_replica.cc:333] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.113308 24421 raft_consensus.cc:2243] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.113832 24421 raft_consensus.cc:2272] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.116380 24421 tablet_replica.cc:333] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.117004 24421 raft_consensus.cc:2243] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.118223 24421 raft_consensus.cc:2272] T 5a3be6e6308145f7a32b2ede29daafa4 P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.121784 24421 tablet_replica.cc:333] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.122696 24421 raft_consensus.cc:2243] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.123767 24421 raft_consensus.cc:2272] T dcb163fdfe074544b7eb876f26ec6570 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.126348 24421 tablet_replica.cc:333] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.126977 24421 raft_consensus.cc:2243] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.127597 24421 raft_consensus.cc:2272] T 3745b69c0304470b91ff96889e8485f2 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.129819 24421 tablet_replica.cc:333] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.130370 24421 raft_consensus.cc:2243] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.131229 24421 raft_consensus.cc:2272] T d385031a46bf491cabbcc60263ac6e9a P 50159f6246ed4d19bc4169a9c142b98d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.133526 24421 tablet_replica.cc:333] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.134017 24421 raft_consensus.cc:2243] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.134469 24421 raft_consensus.cc:2272] T 722efbbd5c784722a3277e0fb9453f21 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.136807 24421 tablet_replica.cc:333] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.137305 24421 raft_consensus.cc:2243] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.137753 24421 raft_consensus.cc:2272] T d2ebc79e324843fc9926cf5699e20916 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.140913 24421 tablet_replica.cc:333] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.141826 24421 raft_consensus.cc:2243] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.142767 24421 raft_consensus.cc:2272] T 51aba7e903494170beedc9ed8d56e026 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.146262 24421 tablet_replica.cc:333] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.147111 24421 raft_consensus.cc:2243] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.148737 24421 raft_consensus.cc:2272] T 26a23c16600842d39941c8d6416ebf54 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.151504 24421 tablet_replica.cc:333] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.152050 24421 raft_consensus.cc:2243] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.152642 24421 raft_consensus.cc:2272] T fc68d9f3528b479baf8a639760570f70 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.155304 24421 tablet_replica.cc:333] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.155773 24421 raft_consensus.cc:2243] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.156653 24421 raft_consensus.cc:2272] T a7cac0b3ada94e2ab2953640133556f7 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.159063 24421 tablet_replica.cc:333] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.159550 24421 raft_consensus.cc:2243] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.160012 24421 raft_consensus.cc:2272] T 291c358aca9c4f399b45bfc8f1884c32 P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.162909 24421 tablet_replica.cc:333] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.163733 24421 raft_consensus.cc:2243] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.165709 24421 raft_consensus.cc:2272] T a960b1c7238842a5b31e0f5d208287cb P 50159f6246ed4d19bc4169a9c142b98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.168248 24421 tablet_replica.cc:333] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d: stopping tablet replica
I20260709 12:23:20.169109 24421 raft_consensus.cc:2243] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.169620 24421 raft_consensus.cc:2272] T f936823aef084ec084d006f0af36a144 P 50159f6246ed4d19bc4169a9c142b98d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:20.207625  4264 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 15 similar messages]
W20260709 12:23:20.213505  4264 consensus_peers.cc:597] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 -> Peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Couldn't send request to peer 50159f6246ed4d19bc4169a9c142b98d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:36539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.223688 24421 tablet_server.cc:196] TabletServer@127.23.217.65:0 shutdown complete.
W20260709 12:23:20.239056  4338 consensus_peers.cc:597] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Couldn't send request to peer 50159f6246ed4d19bc4169a9c142b98d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:36539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:20.245157  4338 consensus_peers.cc:597] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Couldn't send request to peer 50159f6246ed4d19bc4169a9c142b98d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:36539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:20.248432  4264 consensus_peers.cc:597] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 -> Peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Couldn't send request to peer 50159f6246ed4d19bc4169a9c142b98d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:36539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.275666 24421 tablet_server.cc:179] TabletServer@127.23.217.66:0 shutting down...
W20260709 12:23:20.301517  4264 consensus_peers.cc:597] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 -> Peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Couldn't send request to peer 50159f6246ed4d19bc4169a9c142b98d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:36539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:20.309192  4338 consensus_peers.cc:597] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Couldn't send request to peer 50159f6246ed4d19bc4169a9c142b98d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:36539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.321944 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:20.322891 24421 tablet_replica.cc:333] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.323933 24421 raft_consensus.cc:2243] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.324998 24421 raft_consensus.cc:2272] T 752c55dfc1984731b3f2604f63ccae71 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.327833 24421 tablet_replica.cc:333] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.328570 24421 raft_consensus.cc:2243] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.329586 24421 raft_consensus.cc:2272] T c40760ebaa0e422c881e8efcbcbe83d4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.332078 24421 tablet_replica.cc:333] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.332729 24421 raft_consensus.cc:2243] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.333276 24421 raft_consensus.cc:2272] T 97ecb652aac54a2fbb426cd63803e74a P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.335811 24421 tablet_replica.cc:333] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.336402 24421 raft_consensus.cc:2243] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.337329 24421 raft_consensus.cc:2272] T d2ebc79e324843fc9926cf5699e20916 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.339769 24421 tablet_replica.cc:333] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.340488 24421 raft_consensus.cc:2243] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.341485 24421 raft_consensus.cc:2272] T 2263f966bfcd4876ad9943d998fab653 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.344269 24421 tablet_replica.cc:333] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.345000 24421 raft_consensus.cc:2243] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.345647 24421 raft_consensus.cc:2272] T dcb163fdfe074544b7eb876f26ec6570 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.348474 24421 tablet_replica.cc:333] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.349162 24421 raft_consensus.cc:2243] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.349686 24421 raft_consensus.cc:2272] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.352023 24421 tablet_replica.cc:333] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.352618 24421 raft_consensus.cc:2243] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.353088 24421 raft_consensus.cc:2272] T 5a3be6e6308145f7a32b2ede29daafa4 P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.355430 24421 tablet_replica.cc:333] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.356063 24421 raft_consensus.cc:2243] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.356627 24421 raft_consensus.cc:2272] T 3745b69c0304470b91ff96889e8485f2 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.358999 24421 tablet_replica.cc:333] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.359597 24421 raft_consensus.cc:2243] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.360291 24421 raft_consensus.cc:2272] T d385031a46bf491cabbcc60263ac6e9a P 53ebf9844ebb4152846b6e9fd1249a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.362763 24421 tablet_replica.cc:333] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.363423 24421 raft_consensus.cc:2243] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.363919 24421 raft_consensus.cc:2272] T 722efbbd5c784722a3277e0fb9453f21 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.366294 24421 tablet_replica.cc:333] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.366935 24421 raft_consensus.cc:2243] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.367466 24421 raft_consensus.cc:2272] T 26a23c16600842d39941c8d6416ebf54 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.369849 24421 tablet_replica.cc:333] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.370712 24421 raft_consensus.cc:2243] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.371802 24421 raft_consensus.cc:2272] T 51aba7e903494170beedc9ed8d56e026 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.375176 24421 tablet_replica.cc:333] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.376029 24421 raft_consensus.cc:2243] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.376787 24421 raft_consensus.cc:2272] T a960b1c7238842a5b31e0f5d208287cb P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:20.378476  4338 consensus_peers.cc:597] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909): Couldn't send request to peer 53ebf9844ebb4152846b6e9fd1249a17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.379992 24421 tablet_replica.cc:333] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.380685 24421 raft_consensus.cc:2243] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.381203 24421 raft_consensus.cc:2272] T f936823aef084ec084d006f0af36a144 P 53ebf9844ebb4152846b6e9fd1249a17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.384586 24421 tablet_replica.cc:333] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.385324 24421 raft_consensus.cc:2243] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.386651 24421 raft_consensus.cc:2272] T 291c358aca9c4f399b45bfc8f1884c32 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:20.388175  4338 consensus_peers.cc:597] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909): Couldn't send request to peer 53ebf9844ebb4152846b6e9fd1249a17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.389657 24421 tablet_replica.cc:333] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.390255 24421 raft_consensus.cc:2243] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.391250 24421 raft_consensus.cc:2272] T fc68d9f3528b479baf8a639760570f70 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.393581 24421 tablet_replica.cc:333] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17: stopping tablet replica
I20260709 12:23:20.394166 24421 raft_consensus.cc:2243] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.394613 24421 raft_consensus.cc:2272] T a7cac0b3ada94e2ab2953640133556f7 P 53ebf9844ebb4152846b6e9fd1249a17 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.443032 24421 tablet_server.cc:196] TabletServer@127.23.217.66:0 shutdown complete.
W20260709 12:23:20.443408  4338 consensus_peers.cc:597] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909): Couldn't send request to peer 53ebf9844ebb4152846b6e9fd1249a17. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:42909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:20.477048  4338 consensus_peers.cc:597] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 50159f6246ed4d19bc4169a9c142b98d (127.23.217.65:36539): Couldn't send request to peer 50159f6246ed4d19bc4169a9c142b98d. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.65:36539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.496608 24421 tablet_server.cc:179] TabletServer@127.23.217.67:0 shutting down...
W20260709 12:23:20.533149  4338 consensus_peers.cc:597] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 -> Peer 53ebf9844ebb4152846b6e9fd1249a17 (127.23.217.66:42909): Couldn't send request to peer 53ebf9844ebb4152846b6e9fd1249a17. Status: Network error: Client connection negotiation failed: client connection to 127.23.217.66:42909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:20.535984 24421 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:20.536900 24421 tablet_replica.cc:333] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.537979 24421 raft_consensus.cc:2243] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.538627 24421 raft_consensus.cc:2272] T 752c55dfc1984731b3f2604f63ccae71 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.541224 24421 tablet_replica.cc:333] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.541919 24421 raft_consensus.cc:2243] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.542605 24421 raft_consensus.cc:2272] T c40760ebaa0e422c881e8efcbcbe83d4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.545078 24421 tablet_replica.cc:333] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.545635 24421 raft_consensus.cc:2243] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.546123 24421 raft_consensus.cc:2272] T 97ecb652aac54a2fbb426cd63803e74a P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.548771 24421 tablet_replica.cc:333] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.549383 24421 raft_consensus.cc:2243] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.549877 24421 raft_consensus.cc:2272] T 2263f966bfcd4876ad9943d998fab653 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.552179 24421 tablet_replica.cc:333] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.552717 24421 raft_consensus.cc:2243] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.553695 24421 raft_consensus.cc:2272] T 6d7e9dd20dd6474e9fdfbd4a4b4fbb73 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.556216 24421 tablet_replica.cc:333] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.556783 24421 raft_consensus.cc:2243] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.557238 24421 raft_consensus.cc:2272] T 5a3be6e6308145f7a32b2ede29daafa4 P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.559650 24421 tablet_replica.cc:333] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.560232 24421 raft_consensus.cc:2243] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.560724 24421 raft_consensus.cc:2272] T dcb163fdfe074544b7eb876f26ec6570 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.563030 24421 tablet_replica.cc:333] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.563634 24421 raft_consensus.cc:2243] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.564146 24421 raft_consensus.cc:2272] T d2ebc79e324843fc9926cf5699e20916 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.566594 24421 tablet_replica.cc:333] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.567173 24421 raft_consensus.cc:2243] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.567617 24421 raft_consensus.cc:2272] T 291c358aca9c4f399b45bfc8f1884c32 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.569959 24421 tablet_replica.cc:333] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.570506 24421 raft_consensus.cc:2243] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.571028 24421 raft_consensus.cc:2272] T fc68d9f3528b479baf8a639760570f70 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.573354 24421 tablet_replica.cc:333] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.573889 24421 raft_consensus.cc:2243] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.574344 24421 raft_consensus.cc:2272] T a7cac0b3ada94e2ab2953640133556f7 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.576572 24421 tablet_replica.cc:333] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.577157 24421 raft_consensus.cc:2243] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.577646 24421 raft_consensus.cc:2272] T 26a23c16600842d39941c8d6416ebf54 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.580018 24421 tablet_replica.cc:333] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.580518 24421 raft_consensus.cc:2243] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.580986 24421 raft_consensus.cc:2272] T 51aba7e903494170beedc9ed8d56e026 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.583186 24421 tablet_replica.cc:333] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.583685 24421 raft_consensus.cc:2243] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.584607 24421 raft_consensus.cc:2272] T 3745b69c0304470b91ff96889e8485f2 P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.587013 24421 tablet_replica.cc:333] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.587533 24421 raft_consensus.cc:2243] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.588075 24421 raft_consensus.cc:2272] T d385031a46bf491cabbcc60263ac6e9a P 3e5c80c10c344270b0d22ed21ccb1274 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.590384 24421 tablet_replica.cc:333] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.590924 24421 raft_consensus.cc:2243] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.592010 24421 raft_consensus.cc:2272] T 722efbbd5c784722a3277e0fb9453f21 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.594696 24421 tablet_replica.cc:333] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.595342 24421 raft_consensus.cc:2243] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.595860 24421 raft_consensus.cc:2272] T a960b1c7238842a5b31e0f5d208287cb P 3e5c80c10c344270b0d22ed21ccb1274 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.598304 24421 tablet_replica.cc:333] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274: stopping tablet replica
I20260709 12:23:20.598874 24421 raft_consensus.cc:2243] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.599869 24421 raft_consensus.cc:2272] T f936823aef084ec084d006f0af36a144 P 3e5c80c10c344270b0d22ed21ccb1274 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.646158 24421 tablet_server.cc:196] TabletServer@127.23.217.67:0 shutdown complete.
I20260709 12:23:20.683981 24421 master.cc:562] Master@127.23.217.126:34683 shutting down...
I20260709 12:23:20.717484 24421 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:20.718227 24421 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.718640 24421 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7987a990f5aa4865bf369f5ef381659a: stopping tablet replica
I20260709 12:23:20.742750 24421 master.cc:584] Master@127.23.217.126:34683 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (21449 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (42768 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (438136 ms total)
[  PASSED  ] 8 tests.
I20260709 12:23:20.813100 24421 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2225 messages since previous log ~32 seconds ago
I20260709 12:23:20.813381 24421 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2225 messages since previous log ~32 seconds ago
I20260709 12:23:20.813580 24421 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2225 messages since previous log ~32 seconds ago
I20260709 12:23:20.813822 24421 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:23:20.814002 24421 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:23:20.814374 24421 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 10 messages since previous log ~0 seconds ago
I20260709 12:23:20.814723 24421 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~20 seconds ago
